ProteinGym clinical zero-shot indels AUC summary at commit 144fe22; 144fe22b07dfaeec2b366f2346203a9838a55b4c

What this source contributed

Records in this release that cite this source. A record can cite several sources.

1 method
27 configurations
1 protocol
18 evaluations
18 results
1 evidence claim

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Value	Metric	Model or configuration	Dataset	Evaluation
0.663 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Hidden Markov Model [ProteinGym 144fe22 clinical summary]	ProteinGym clinical indel benchmark at commit 144fe22	Hidden Markov Model: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.935 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	PoET (200M) [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	PoET (200M): ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.844 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Progen2 Base [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Progen2 Base: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.851 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Progen2 L [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Progen2 L: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.846 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Progen2 M [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Progen2 M: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.763 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Progen2 S [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Progen2 S: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.842 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Progen2 XL [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Progen2 XL: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.649 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	ProtGPT2 [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	ProtGPT2: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.927 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Provean [ProteinGym 144fe22 clinical summary]	ProteinGym clinical indel benchmark at commit 144fe22	Provean: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.922 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	RITA L [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	RITA L: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.892 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	RITA M [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	RITA M: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.784 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	RITA S [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	RITA S: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.916 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	RITA XL [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	RITA XL: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.852 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Tranception L no retrieval [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Tranception L no retrieval: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.843 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Tranception L [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Tranception L: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.831 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Tranception M no retrieval [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Tranception M no retrieval: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.828 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Tranception M [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Tranception M: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)
0.599 unitless	AUROC (ProteinGym clinical indels, zero-shot, Average_AUC over proteins)	Unirep [ProteinGym 144fe22b snapshot]	ProteinGym clinical indel benchmark at commit 144fe22	Unirep: ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)

Claims

Statements this source supports, each with where in the source it was checked.

Relevance of vep-lit-20261014-protocol-proteingym-clinical-zero-shot-indels-auc-144fe22 to use-case-rare-disease-candidate-ranking: Assessed by ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22) (vep-lit-20261014-source-proteingym-clinical-zero-shot-indels-auc-summary-144fe22: Summary_performance_clinical_indels_AUC.csv; README line 22)

Models, benchmarks and other records

Methods (1)

Profile hidden Markov model (ProteinGym baseline)

Configurations (27)

Protocols (1)

ProteinGym clinical indels, zero-shot, average AUC (commit 144fe22)

Evaluations (18)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

26 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
attributes.artifact_sha256 98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836 Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_sha256` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Artifact url https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_url` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Licence MIT (repository LICENSE); underlying assay and clinical data carry their own terms Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.licence` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Limitations 2 values Mutable repository: the file is pinned to commit 144fe22b07dfaeec2b366f2346203a9838a55b4c, which was the head of main at retrieval (committed 2026-03-25). The repository does not record which script run, data release or model-score snapshot produced this file; the README says these summary files are what the website hosts. Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.limitations` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Media type text/csv Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.media_type` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Publication status official_project_source Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.publication_status` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieved at 2026-10-14T13:39:33Z Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.retrieved_at` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.review.artifact_sha256 98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836 Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.artifact_sha256` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review: method note Re-downloaded from the recorded URL on 2026-10-14 and matched the SHA-256; URL, version, licence and limitations checked. Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.method_note` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review method 2 values source-hash-verification ai-assisted-source-review Source metadata	ProteinGym clinical zero-shot indels AUC summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:33Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.method` Source artifact SHA-256: `98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Official ProteinGym summary file: average AUC on the clinical indel benchmark for 18 zero-shot models.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: vep-lit-20261014-source-proteingym-clinical-zero-shot-indels-auc-summary-144fe22

areas: proteins-complexes
contexts: clinical_research
url: https://github.com/OATML-Markslab/ProteinGym/blob/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv
artifact url: https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv
version: 144fe22b07dfaeec2b366f2346203a9838a55b4c
retrieved at: 2026-10-14T13:39:33Z
artifact sha256: 98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836
publication status: official_project_source
licence: MIT (repository LICENSE); underlying assay and clinical data carry their own terms
media type: text/csv
source locator: benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv
limitations: Mutable repository: the file is pinned to commit 144fe22b07dfaeec2b366f2346203a9838a55b4c, which was the head of main at retrieval (committed 2026-03-25).; The repository does not record which script run, data release or model-score snapshot produced this file; the README says these summary files are what the website hosts.
review: reviewer: claude; reviewer note: Claude review agent; no human review claimed. Independent Claude (Opus 5.5) review agent, separate from the extraction agent that built this batch. Automated review only. Transcription, not reproduction.; reviewed at: 2026-10-14T15:30:00Z; artifact sha256: 98a532c0e2c3d4720b73b7c05cf0dcf0419438c326761aaae3b788799f5aa836; retrieval url: https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/clinical_zero_shot/indels/AUC/Summary_performance_clinical_indels_AUC.csv; method: source-hash-verification; ai-assisted-source-review; method note: Re-downloaded from the recorded URL on 2026-10-14 and matched the SHA-256; URL, version, licence and limitations checked.; note: Hash matches.

Related records