ProteinGym clinical indel variants (benign or pathogenic labels per protein); the README at this commit gives 1,555 clinical proteins.
18 evaluations · 18 results. Different protocols are not a single leaderboard.
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: PoET (200M) [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.935 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourcePoET (200M): ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 2, row 'PoET (200M)', column 'Average_AUC' |
| Configuration: Progen2 Base [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.844 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProgen2 Base: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 10, row 'Progen2 Base', column 'Average_AUC' |
| Configuration: Progen2 L [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.851 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProgen2 L: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 8, row 'Progen2 L', column 'Average_AUC' |
| Configuration: Progen2 M [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.846 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProgen2 M: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 9, row 'Progen2 M', column 'Average_AUC' |
| Configuration: Progen2 S [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.763 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProgen2 S: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 16, row 'Progen2 S', column 'Average_AUC' |
| Configuration: Progen2 XL [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.842 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProgen2 XL: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 12, row 'Progen2 XL', column 'Average_AUC' |
| Configuration: Hidden Markov Model [ProteinGym 144fe22 clinical summary] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.663 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceHidden Markov Model: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 17, row 'Hidden Markov Model', column 'Average_AUC' |
| Configuration: Provean [ProteinGym 144fe22 clinical summary] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.927 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProvean: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 3, row 'Provean', column 'Average_AUC' |
| Configuration: ProtGPT2 [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.649 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceProtGPT2: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 18, row 'ProtGPT2', column 'Average_AUC' |
| Configuration: RITA L [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.922 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceRITA L: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 4, row 'RITA L', column 'Average_AUC' |
| Configuration: RITA M [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.892 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceRITA M: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 6, row 'RITA M', column 'Average_AUC' |
| Configuration: RITA S [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.784 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceRITA S: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 15, row 'RITA S', column 'Average_AUC' |
| Configuration: RITA XL [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.916 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceRITA XL: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 5, row 'RITA XL', column 'Average_AUC' |
| Configuration: Tranception L [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.843 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceTranception L: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 11, row 'Tranception L', column 'Average_AUC' |
| Configuration: Tranception L no retrieval [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.852 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceTranception L no retrieval: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 7, row 'Tranception L no retrieval', column 'Average_AUC' |
| Configuration: Tranception M [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.828 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceTranception M: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 14, row 'Tranception M', column 'Average_AUC' |
| Configuration: Tranception M no retrieval [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.831 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceTranception M no retrieval: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 13, row 'Tranception M no retrieval', column 'Average_AUC' |
| Configuration: Unirep [ProteinGym 144fe22b snapshot] | Protocol: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Dataset subset: ProteinGym clinical indel benchmark at commit 144fe22 | 0.599 AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Evaluation origin not reported · Source checkedMethods, coverage and sourceUnirep: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) Aggregation: Not reported ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 · benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv line 19, row 'Unirep', column 'Average_AUC' |
Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.
0 of 4 readiness checks met. These checks assess whether the evidence supports a reproducible investigation; a source-checked score alone does not meet them.
Release 2026-10-22-1b7cf027e4f6 · Evidence verified: Not verified
Exact outcomes, predictions, identifiers and evaluator are connected.
Missing or unresolved evidence
Verified: Not verified
Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.
Missing or unresolved evidence
Verified: Not verified
A pinned recipe describes the inputs, environment and resource requirements.
Missing or unresolved evidence
Verified: Not verified
Separate data and exposure records support an independent test.
Missing or unresolved evidence
Verified: Not verified
Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.
No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.
This record describes a particular subset or cohort used in an evaluation. Its results do not describe the full dataset.
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
5 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Source location README.md line 22 Context-only references | OATML-Markslab/ProteinGym / README.md README.md line 22 Version: Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Subset scope 1,555 clinical proteins (README line 22). The README gives protein counts only, not variant counts. Context-only references | OATML-Markslab/ProteinGym / README.md README.md line 22 Version: Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Version 144fe22b07dfaeec2b366f2346203a9838a55b4c Context-only references | OATML-Markslab/ProteinGym / README.md README.md line 22 Version: Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description ProteinGym clinical indel variants (benign or pathogenic labels per protein); the README at this commit gives 1,555 clinical proteins. Context-only references | OATML-Markslab/ProteinGym / README.md README.md line 22 Version: Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Name ProteinGym clinical indel benchmark at commit 144fe22 Context-only references | OATML-Markslab/ProteinGym / README.md README.md line 22 Version: Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
Stable ID: vep-lit-20261014-data-proteingym-clinical-indels-144fe22