Model

CARP

Convolutional protein language model pretrained by masked language modelling on protein sequences, scored in ProteinGym as a zero-shot single-sequence substitution baseline.

8 evaluations · 76 results 1 configuration with further results

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Evaluations and results

8 evaluations · 76 results. Different protocols are not a single leaderboard.

Sorted by Spearman (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	Protocol and dataset	Spearman	Spearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays))	Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, variants with 3 substitutions: mean over protein and selection-type groups of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, variants with 4 substitutions: mean over protein and selection-type groups of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, variants with 5 or more substitutions: mean over protein and selection-type groups of assay-level Spearman)	Spearman (ProteinGym zero-shot DMS substitutions, Virus proteins: mean over proteins of assay-level Spearman)
CARP (640M) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym AMFR official zero-shot Spearman snapshot, ProteinGym AMFR 4G3O official snapshot cohort	0.165 (best in column)	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
CARP (76M) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym AMFR official zero-shot Spearman snapshot, ProteinGym AMFR 4G3O official snapshot cohort	0.114	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
CARP (38M) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym AMFR official zero-shot Spearman snapshot, ProteinGym AMFR 4G3O official snapshot cohort	-0.185	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
CARP (600K) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym AMFR official zero-shot Spearman snapshot, ProteinGym AMFR 4G3O official snapshot cohort	-0.194	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
CARP (38M) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym v1.3 zero-shot DMS substitutions, ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	Not reported	0.285	0.268	0.312	0.354	0.322	0.206	0.28	0.24	0.218	0.307	0.253	0.314	0.279	0.173	0.106	0.126	0.143	0.127
CARP (600K) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym v1.3 zero-shot DMS substitutions, ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	Not reported	0.112	0.084	0.171	0.1	0.122	0.101	0.106	0.085	0.059	0.086	0.075	0.105	0.108	0.029	0.045	0.076	0.096	0.055
CARP (640M) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym v1.3 zero-shot DMS substitutions, ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	Not reported	0.395 (best in column)	0.273	0.397 (best in column)	0.424 (best in column)	0.417 (best in column)	0.331 (best in column)	0.369 (best in column)	0.374 (best in column)	0.366 (best in column)	0.395 (best in column)	0.381 (best in column)	0.412 (best in column)	0.391 (best in column)	0.213 (best in column)	0.172 (best in column)	0.159 (best in column)	0.162 (best in column)	0.284 (best in column)
CARP (76M) [ProteinGym 144fe22b snapshot]Evaluation	ProteinGym v1.3 zero-shot DMS substitutions, ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	Not reported	0.342	0.282 (best in column)	0.369	0.403	0.388	0.254	0.328	0.302	0.271	0.366	0.301	0.376	0.336	0.206	0.115	0.124	0.127	0.157

All 76 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym AMFR official zero-shot Spearman snapshot Dataset subset: ProteinGym AMFR 4G3O official snapshot cohort	-0.185 Spearman unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M) AMFR official snapshot Not reported Aggregation: Not reported ProteinGym AMFR assay-level Spearman comparison · DMS_substitutions_Spearman_DMS_level.csv line 15, DMS ID=AMFR_HUMAN_Tsuboyama_2023_4G3O, column=CARP (38M)
Configuration: CARP (600K) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym AMFR official zero-shot Spearman snapshot Dataset subset: ProteinGym AMFR 4G3O official snapshot cohort	-0.194 Spearman unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (600K) AMFR official snapshot Not reported Aggregation: Not reported ProteinGym AMFR assay-level Spearman comparison · DMS_substitutions_Spearman_DMS_level.csv line 15, DMS ID=AMFR_HUMAN_Tsuboyama_2023_4G3O, column=CARP (600K)
Configuration: CARP (640M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym AMFR official zero-shot Spearman snapshot Dataset subset: ProteinGym AMFR 4G3O official snapshot cohort	0.165 Spearman unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (640M) AMFR official snapshot Not reported Aggregation: Not reported ProteinGym AMFR assay-level Spearman comparison · DMS_substitutions_Spearman_DMS_level.csv line 15, DMS ID=AMFR_HUMAN_Tsuboyama_2023_4G3O, column=CARP (640M)
Configuration: CARP (76M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym AMFR official zero-shot Spearman snapshot Dataset subset: ProteinGym AMFR 4G3O official snapshot cohort	0.114 Spearman unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (76M) AMFR official snapshot Not reported Aggregation: Not reported ProteinGym AMFR assay-level Spearman comparison · DMS_substitutions_Spearman_DMS_level.csv line 15, DMS ID=AMFR_HUMAN_Tsuboyama_2023_4G3O, column=CARP (76M)
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.28 Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays)) unitless · higher Uncertainty: SE 0.015. From compute_bootstrap_standard_error_functional_categories in performance_DMS_benchmarks.py at 144fe22 (default 10,000 resamples, lines 95-111 and 310-315); not an absolute standard error of the model's own average. Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Average_Spearman'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.279 Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Depth_1'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.173 Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Depth_2'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.106 Spearman (ProteinGym zero-shot DMS substitutions, variants with 3 substitutions: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Depth_3'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.126 Spearman (ProteinGym zero-shot DMS substitutions, variants with 4 substitutions: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Depth_4'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.143 Spearman (ProteinGym zero-shot DMS substitutions, variants with 5 or more substitutions: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Depth_5+'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.285 Spearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Function_Activity'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.268 Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Function_Binding'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.312 Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Function_Expression'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.218 Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Function_OrganismalFitness'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.314 Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Function_Stability'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.354 Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'High_MSA_depth'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.206 Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Low_MSA_depth'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.24 Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Medium_MSA_depth'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.322 Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Taxa_Human'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.307 Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Taxa_Other_Eukaryote'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.253 Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Taxa_Prokaryote'
Configuration: CARP (38M) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.127 Spearman (ProteinGym zero-shot DMS substitutions, Virus proteins: mean over proteins of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 91, row 'CARP (38M)', column 'Taxa_Virus'
Configuration: CARP (600K) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.106 Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays)) unitless · higher Uncertainty: SE 0.016. From compute_bootstrap_standard_error_functional_categories in performance_DMS_benchmarks.py at 144fe22 (default 10,000 resamples, lines 95-111 and 310-315); not an absolute standard error of the model's own average. Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (600K): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 97, row 'CARP (600K)', column 'Average_Spearman'
Configuration: CARP (600K) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.108 Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (600K): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 97, row 'CARP (600K)', column 'Depth_1'
Configuration: CARP (600K) [ProteinGym 144fe22b snapshot]	Protocol: ProteinGym v1.3 zero-shot DMS substitutions Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	0.029 Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Evaluation origin not reported · Source checked Methods, coverage and source CARP (600K): ProteinGym zero-shot DMS substitutions summary (144fe22) ProteinGym zero-shot DMS substitutions, Spearman summary Aggregation: Not reported ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 97, row 'CARP (600K)', column 'Depth_2'

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Further results on linked records

One linked record has results that are not included above, because no reviewed claim yet ties the link to this model. Each record keeps its own settings and evaluations, and its scores are on its own page.

CARP (Livesey and Marsh 2025) Configuration · configuration · 5 evaluations · 6 results

Use this model

How it works, versions and access

Versions and evaluated configurations

CARP (38M) [ProteinGym 144fe22b snapshot] · configuration of
CARP (600K) [ProteinGym 144fe22b snapshot] · configuration of
CARP (640M) [ProteinGym 144fe22b snapshot] · configuration of
CARP (76M) [ProteinGym 144fe22b snapshot] · configuration of

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

3 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
Description Convolutional protein language model pretrained by masked language modelling on protein sequences, scored in ProteinGym as a zero-shot single-sequence substitution baseline. Individual claims	Abstract: Convolutions are competitive with transformers for protein sequence pretraining Original source ↗ uc20260930-source-proteingym-amfr-spearman: Benchmark CSV header (model scored in the zero-shot DMS substitution benchmark); run-doc-proteingym-readme-md-144fe22b: README 'Leaderboard' baseline table (model name, input modalities, reference); qa-descriptions-20261010-source-carp: abstract Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Europe PMC core record MED/38428432 (title and abstract), Cell systems 2024 Retrieved: 2026-10-10T06:44:10Z	source checked ["automated-source-review","ai-assisted-source-review"] · 2026-10-10 Audit details Description written from the cited source text; provenance text was generic and is kept here as previous_value. Independent review 2026-10-10: every stated fact is supported by the cited text. Field: `description` Claim: metadata-correction-6c29cd67ac588974bbf0 Source artifact SHA-256: `68e3e9f21b77774911d973330d658ce6dafd42634337718da56c24a780caf90d` Hash scope: SHA-256 of one retrieval of a Europe PMC REST search response. The response is not byte-stable: on 2026-10-10 a second retrieval of 10 of these 31 records differed only in JSON key order and parsed to identical content. The title and abstractText fields are the evidence. Inspected artifact
Description Convolutional protein language model pretrained by masked language modelling on protein sequences, scored in ProteinGym as a zero-shot single-sequence substitution baseline. Individual claims	OATML-Markslab/ProteinGym / README.md Original source ↗ uc20260930-source-proteingym-amfr-spearman: Benchmark CSV header (model scored in the zero-shot DMS substitution benchmark); run-doc-proteingym-readme-md-144fe22b: README 'Leaderboard' baseline table (model name, input modalities, reference); qa-descriptions-20261010-source-carp: abstract Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-09-17T09:32:03.517591+00:00	source checked ["automated-source-review","ai-assisted-source-review"] · 2026-10-10 Audit details Description written from the cited source text; provenance text was generic and is kept here as previous_value. Independent review 2026-10-10: every stated fact is supported by the cited text. Field: `description` Claim: metadata-correction-6c29cd67ac588974bbf0 Source artifact SHA-256: `321487a8de52c6cfa647a658f61150dd72e0acb0125470524fb94d1f8b23321a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Description Convolutional protein language model pretrained by masked language modelling on protein sequences, scored in ProteinGym as a zero-shot single-sequence substitution baseline. Individual claims	ProteinGym AMFR assay-level Spearman comparison Original source ↗ uc20260930-source-proteingym-amfr-spearman: Benchmark CSV header (model scored in the zero-shot DMS substitution benchmark); run-doc-proteingym-readme-md-144fe22b: README 'Leaderboard' baseline table (model name, input modalities, reference); qa-descriptions-20261010-source-carp: abstract Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-09-30	source checked ["automated-source-review","ai-assisted-source-review"] · 2026-10-10 Audit details Description written from the cited source text; provenance text was generic and is kept here as previous_value. Independent review 2026-10-10: every stated fact is supported by the cited text. Field: `description` Claim: metadata-correction-6c29cd67ac588974bbf0 Source artifact SHA-256: `f432423b87f79ac9778dfac86e3d95be041d246bc618dc0a406b35b0b7466437` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

3 source records and release history

ProteinGym AMFR assay-level Spearman comparison · Original source · 144fe22b07dfaeec2b366f2346203a9838a55b4c
OATML-Markslab/ProteinGym / README.md · Original source · Git commit 144fe22b07dfaeec2b366f2346203a9838a55b4c
Abstract: Convolutions are competitive with transformers for protein sequence pretraining · Original source · Europe PMC core record MED/38428432 (title and abstract), Cell systems 2024

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: uc20260930-model-carp

areas: proteins-complexes
source locator: Pinned ProteinGym Spearman CSV header: CARP (38M)
review: method: automated-source-review; reviewer: codex; reviewed at: 2026-09-30T21:40:59.576893+00:00; scope: Primary-source transcription and catalogue applicability audit; not human scientific review or independent experimental reproduction.; reviewer note: role: experimental use-case curation
entity level: family
identity scope: Family/method identity only; no undocumented checkpoint equivalence.

Related records

subject: CARP: description
configuration of: CARP (38M) [ProteinGym 144fe22b snapshot]
configuration of: CARP (600K) [ProteinGym 144fe22b snapshot]
configuration of: CARP (640M) [ProteinGym 144fe22b snapshot]
configuration of: CARP (76M) [ProteinGym 144fe22b snapshot]
configuration of: CARP (Livesey and Marsh 2025)

Suggest a correction