Datasets
CATHe-derived large and small superfamily datasets based on CATH v4.3 and CATH-Gene3D.
Task
CATH superfamily annotation
CATH superfamily annotation evaluates protein-domain classification with stringent sequence-identity separation.
Sources
CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passagesOverview
Metrics
F1 is emphasized for class imbalance; balanced accuracy and MCC are also computed.
Allowed inputs
Amino-acid embeddings, 3Di embeddings, or their concatenation.
Sources
CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passagesEvaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passageslimited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
1 evaluation · 1 result. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Pipeline: CATHe2 + ProstT5 | Task: CATH superfamily annotation Dataset: CATH superfamily benchmark | 82.3% F1 percent · unknown Uncertainty: ± 1.3 percentage points Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceCATHe2 + ProstT5: CATH superfamily annotation amino-acid and structural alphabet embedding classifier Aggregation: Not reported CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Table 3, ProstT5 full row, F1 score column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
CATHe-derived large and small superfamily datasets based on CATH v4.3 and CATH-Gene3D. Training, validation and test domain sets are retained from CATHe where structural-token availability permits. F1 is emphasized for class imbalance; balanced accuracy and MCC are also computed. CATHe and alternative embeddings from ProtT5, ProstT5, ESM2, Ankh and TM-Vec; domain availability and input dimensionality differ. The source reports less than 20% sequence identity within and between the original partitions. Bootstrap resampling of the test set supplies interval estimates.
Sources
CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passagesRecorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
- The source documents training-domain losses caused by structural-token availability, exposing a key comparison confound.
Sources
CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages
Limitations and conditions
- Unavailable structural inputs remove domains and classes from the larger dataset; comparison with the original CATHe corpus must preserve that change in evaluation coverage.
Sources
CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages
Profile review details
Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.
Stable record: reported-task-c98e91ffc7247dSpecifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | CATHe-derived large and small superfamily datasets based on CATH v4.3 and CATH-Gene3D.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Splits | Training, validation and test domain sets are retained from CATHe where structural-token availability permits.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Metrics | F1 is emphasized for class imbalance; balanced accuracy and MCC are also computed.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Baselines | CATHe and alternative embeddings from ProtT5, ProstT5, ESM2, Ankh and TM-Vec; domain availability and input dimensionality differ.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Leakage controls | The source reports less than 20% sequence identity within and between the original partitions.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Uncertainty | Bootstrap resampling of the test set supplies interval estimates.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Entity type | Paper-specific computational evaluation protocol.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Organisms | The benchmark spans CATH superfamilies using PDB and AlphaFold-database domains. Data and datasets specifies structural-domain and superfamily coverage, but does not provide a species inventory; superfamily classes are not organism labels. · Not reported in inspected sourcesSourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Data and datasets; 3Di processing |
| Assays | CATH domain superfamily annotations and structural-alphabet representations.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Allowed inputs | Amino-acid embeddings, 3Di embeddings, or their concatenation.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
| Adaptation | Frozen protein-model embeddings feed a supervised feed-forward classifier; classifier/data-filter hyperparameters are selected in the study.SourcesCATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets | journal full text in PMC | Read source |
Historical gaps recorded on 2026-09-17
The catalogue now holds 1 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- complete numerical transcription and independent cell review: Full primary artifact and table inventory preserved; no new numeric row is published from this audit alone.
- exact checkpoint hashes and per-method scored denominators: Table labels alone do not establish these fields; do not infer checkpoint or scored count from model name or dataset size.
Search and extraction details
primary comparison tables located
Searches
- CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets 10.1093/biomethods/bpaf080
Evidence locations
- Table 1.; XML table bpaf080-T1
- Table 3.; XML table bpaf080-T3
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
18 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets CATHe-derived large and small superfamily datasets based on CATH v4.3 and CATH-Gene3D. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Training, validation and test domain sets are retained from CATHe where structural-token availability permits. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Frozen protein-model embeddings feed a supervised feed-forward classifier; classifier/data-filter hyperparameters are selected in the study. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics F1 is emphasized for class imbalance; balanced accuracy and MCC are also computed. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines CATHe and alternative embeddings from ProtT5, ProstT5, ESM2, Ankh and TM-Vec; domain availability and input dimensionality differ. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls The source reports less than 20% sequence identity within and between the original partitions. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty Bootstrap resampling of the test set supplies interval estimates. Individual claims | CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
2 source records and release history
- CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Original source · journal full text in PMC
- CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets · Original source · journal full text in PMC
Technical metadata and extraction receipts
Stable ID: reported-task-c98e91ffc7247d
- areas
- proteins-complexes
- tasks
- CATH superfamily annotation
- entity level
- task
- version
- Not reported
- task
- CATH superfamily annotation
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: primary_comparison_tables_located; primary sources: evidence-expansion-cathe2-2025-713dbfb6; inspected locators: Table 1.; XML table bpaf080-T1; Table 3.; XML table bpaf080-T3; searched queries: CATHe2: Enhanced CATH superfamily detection using ProstT5 and structural alphabets 10.1093/biomethods/bpaf080; gaps: complete numerical transcription and independent cell review: Full primary artifact and table inventory preserved; no new numeric row is published from this audit alone.; exact checkpoint hashes and per-method scored denominators: Table labels alone do not establish these fields; do not infer checkpoint or scored count from model name or dataset size.; claim scope: Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: cathe2-2025; source locator: Methods: Data and datasets; Results; cached text lines 11–14, 30; comparative evaluation and ablation passages; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.
Related records
- dataset: CATH superfamily benchmark
- benchmark: CATHe2 + ProstT5: CATH superfamily annotation