Livesey and Marsh 2025 VEP rankings
Direct evidenceIndependent studyPairwise win rate (higher is better), with the other metrics below each value. Showing the top 10 of 97 rows. The best value in each column is highlighted.
Scroll sideways to see all 5 columns.
| Tool | DMS benchmark, all 36 DMS sets | DMS benchmark, direct assays only | DMS benchmark, indirect assays only | DMS benchmark, SNV-accessible substitutions | DMS benchmark, SNV-accessible, gaps filled |
|---|---|---|---|---|---|
| CPT-1machine learning | 92.82% (best in column)all other VEPs,Pairwise win rate 66.2% | 93.46% (best in column) | 92.08% (best in column) | 92.13% (best in column) | 91.91% (best in column) |
| AlphaMissensemachine learning | 90.69%all other VEPs,Pairwise win rate 65.3% | 92.14% | 89.50% | 89.20% | 89.46% |
| ESCOTTspecialist model | 88.18%all other VEPs,Pairwise win rate 50.7% | 91.77% | 85.51% | 87.27% | 87.09% |
| popEVEspecialist model | 87.27%all other VEPs,Pairwise win rate 46.3% | 89.19% | 86.10% | 87.89% | 88.12% |
| iGEMMEspecialist model | 86.02%all other VEPs,Pairwise win rate 34.9% | 89.59% | 83.41% | 85.94% | 85.81% |
| GEMMEspecialist model | 84.94%all other VEPs,Pairwise win rate 36.1% | 88.70% | 82.15% | 83.61% | 83.78% |
| SaProtfoundation model | 83.56%all other VEPs,Pairwise win rate 45.2% | 84.80% | 82.72% | 83.39% | 83.22% |
| TranceptEVEspecialist model | 82.08%all other VEPs,Pairwise win rate 25.7% | 88.50% | 78.18% | 83.60% | 75.21% |
| VARITY_Rmachine learning | 81.84%all other VEPs,Pairwise win rate 31.8% | 77.52% | 86.41% | 81.84% | 82.07% |
| ESM-1vfoundation model | 81.49%all other VEPs,Pairwise win rate 28.1% | 78.36% | 83.63% | 81.52% | 81.83% |
| VARITY_ERmachine learning | 80.49%all other VEPs,Pairwise win rate 27.9% | 77.41% | 83.67% | 80.49% | 80.55% |
| mvPPTmachine learning | 77.72%all other VEPs,Pairwise win rate 24.5% | 73.95% | 82.13% | 77.72% | 70.86% |
| SNPredmachine learning | 77.57%all other VEPs,Pairwise win rate 20.8% | 79.06% | 76.39% | 77.57% | 78.13% |
| EVEspecialist model | 77.31%all other VEPs,Pairwise win rate 18.1% | 82.98% | 72.47% | 77.68% | 72.48% |
| DeepSequencespecialist model | 76.57%all other VEPs,Pairwise win rate 17.3% | 73.48% | 78.36% | 78.37% | 77.51% |
| CARPfoundation model | 76.05%all other VEPs,Pairwise win rate 20.0% | 73.05% | 77.69% | 77.16% | 71.21% |
| MSA_Transformerfoundation model | 75.89%all other VEPs,Pairwise win rate 24.4% | 77.72% | 74.67% | 79.24% | 71.98% |
| MetaRNNmachine learning | 73.20%all other VEPs,Pairwise win rate 16.6% | 74.83% | 71.86% | 73.20% | 74.07% |
| ESM2foundation model | 72.22%all other VEPs,Pairwise win rate 27.8% | 74.21% | 70.77% | 70.55% | 71.07% |
| MutFormermachine learning | 71.39%all other VEPs,Pairwise win rate 17.2% | 74.66% | 68.92% | 71.39% | 70.47% |
| VESPAmachine learning | 71.33%all other VEPs,Pairwise win rate 17.3% | 69.53% | 72.57% | 69.48% | 64.84% |
| EVmutation_epistaticspecialist model | 70.82%all other VEPs,Pairwise win rate 10.7% | 74.27% | 68.26% | 71.99% | 68.81% |
| PHACTboostmachine learning | 70.69%all other VEPs,Pairwise win rate 15.3% | 66.84% | 73.43% | 71.87% | 72.51% |
| gMVPmachine learning | 69.88%all other VEPs,Pairwise win rate 23.3% | 61.74% | 75.66% | 69.88% | 70.97% |
| mutationTCNspecialist model | 67.99%all other VEPs,Pairwise win rate 9.21% | 70.73% | 65.29% | 67.85% | 63.44% |
| UNEECONspecialist model | 66.79%all other VEPs,Pairwise win rate 12.5% | 68.09% | 65.67% | 66.79% | 65.15% |
| REVELmachine learning | 66.13%all other VEPs,Pairwise win rate 13.2% | 65.58% | 66.33% | 66.13% | 67.45% |
| ESM-1bfoundation model | 65.04%all other VEPs,Pairwise win rate 14.7% | 62.38% | 66.95% | 64.51% | 65.92% |
| EVmutation_independentspecialist model | 64.45%all other VEPs,Pairwise win rate 8.62% | 67.11% | 62.42% | 64.04% | 63.76% |
| MutScoremachine learning | 63.41%all other VEPs,Pairwise win rate 8.64% | 61.53% | 64.82% | 63.41% | 63.22% |
| ClinPredmachine learning | 63.16%all other VEPs,Pairwise win rate 4.36% | 66.86% | 60.42% | 63.16% | 64.48% |
| DEOGEN2machine learning | 59.39%all other VEPs,Pairwise win rate 11.4% | 66.07% | 54.64% | 59.84% | 61.09% |
| VEST4machine learning | 59.03%all other VEPs,Pairwise win rate 2.33% | 59.86% | 58.44% | 59.04% | 60.51% |
| MutationAssessorspecialist model | 58.80%all other VEPs,Pairwise win rate 10.3% | 59.34% | 58.32% | 58.80% | 57.31% |
| SNAP2machine learning | 58.66%all other VEPs,Pairwise win rate 7.84% | 56.53% | 59.94% | 56.98% | 58.32% |
| SuSPectmachine learning | 58.36%all other VEPs,Pairwise win rate 10.4% | 59.53% | 57.49% | 55.62% | 56.59% |
| SNPs&GOmachine learning | 57.38%all other VEPs,Pairwise win rate 5.91% | 54.96% | 59.04% | 59.02% | 54.23% |
| BayesDel_noAFmachine learning | 57.10%all other VEPs,Pairwise win rate 6% | 56.63% | 57.42% | 57.10% | 58.25% |
| BayesDelmachine learning | 56.92%all other VEPs,Pairwise win rate 5.44% | 56.77% | 57.01% | 56.92% | 58.13% |
| ProGen2foundation model | 56.66%all other VEPs,Pairwise win rate 8.89% | 60.96% | 53.06% | 56.51% | 58.58% |
| PHACTspecialist model | 54.74%all other VEPs,Pairwise win rate 6.71% | 51.62% | 56.96% | 55.24% | 56.41% |
| MISTICmachine learning | 54.51%all other VEPs,Pairwise win rate 7.24% | 57.28% | 52.33% | 54.51% | 55.43% |
| SPRImachine learning | 54.25%all other VEPs,Pairwise win rate 2% | 60.21% | 50.25% | 56.26% | 60.61% |
| MOIpred_benignmachine learning | 54.10%all other VEPs,Pairwise win rate 11.2% | 54.63% | 53.69% | 54.10% | 52.66% |
| DeepSAVmachine learning | 53.68%all other VEPs,Pairwise win rate 6.74% | 54.71% | 52.89% | 53.62% | 55.51% |
| MPCmachine learning | 53.56%all other VEPs,Pairwise win rate 11.6% | 57.19% | 50.68% | 53.56% | 54.78% |
| VESPAlmachine learning | 53.47%all other VEPs,Pairwise win rate 10.9% | 48.83% | 56.61% | 51.64% | 52.53% |
| Rhapsodymachine learning | 53.40%all other VEPs,Pairwise win rate 4.83% | 47.19% | 57.85% | 51.21% | 56.89% |
| PhD_SNPmachine learning | 53.35%all other VEPs,Pairwise win rate 1.99% | 52.24% | 54.01% | 55.62% | 50.74% |
| Eigenspecialist model | 52.37%all other VEPs,Pairwise win rate 4.83% | 53.68% | 51.30% | 52.37% | 53.87% |
| SIFT4Gconventional | 52.16%all other VEPs,Pairwise win rate 6.85% | 49.94% | 53.64% | 52.16% | 53.62% |
| DeMaSkmachine learning | 50.68%all other VEPs,Pairwise win rate 11.1% | 47.93% | 53.25% | 49.24% | 49.84% |
| PolyPhen2_HumVarmachine learning | 48.35%all other VEPs,Pairwise win rate 8.47% | 46.24% | 49.95% | 48.41% | 49.67% |
| PROVEANconventional | 48.07%all other VEPs,Pairwise win rate 5.47% | 50.96% | 45.97% | 48.28% | 49.99% |
| MutPredmachine learning | 47.95%all other VEPs,Pairwise win rate 14.4% | 46.47% | 49.04% | 47.95% | 49.24% |
| CONDELspecialist model | 47.52%all other VEPs,Pairwise win rate 9.76% | 45.21% | 49.04% | 47.52% | 47.78% |
| SNPs&GO3Dmachine learning | 47.51%all other VEPs,Pairwise win rate 7.16% | 44.78% | 49.44% | 47.68% | 47.46% |
| RITAfoundation model | 46.42%all other VEPs,Pairwise win rate 5.80% | 48.23% | 45.37% | 45.21% | 42.63% |
| SIGMAmachine learning | 46.13%all other VEPs,Pairwise win rate 10.6% | 47.69% | 45.16% | 46.13% | 47.10% |
| MetaLRmachine learning | 45.70%all other VEPs,Pairwise win rate 8.13% | 42.05% | 48.26% | 45.70% | 46.62% |
| Envisionmachine learning | 45.23%all other VEPs,Pairwise win rate 7.44% | 49.23% | 42.40% | 45.64% | 46.81% |
| Tranceptionfoundation model | 44.51%all other VEPs,Pairwise win rate 6.44% | 51.89% | 39.26% | 45.15% | 46.56% |
| PANTHERspecialist model | 44.42%all other VEPs,Pairwise win rate 1.86% | 44.14% | 44.57% | 45.68% | 37.66% |
| Wavenetspecialist model | 42.79%all other VEPs,Pairwise win rate 11.1% | 43.33% | 42.41% | 42.77% | 43.08% |
| LASSIEspecialist model | 42.67%all other VEPs,Pairwise win rate 1.22% | 45.21% | 40.97% | 42.67% | 43.78% |
| PolyPhen2_HumDivmachine learning | 42.41%all other VEPs,Pairwise win rate 6.76% | 41.50% | 43.11% | 42.41% | 44.12% |
| Maverick_benignmachine learning | 41.20%all other VEPs,Pairwise win rate 6.78% | 41.09% | 41.31% | 41.20% | 42.66% |
| Unirep_evotunefoundation model | 40.99%all other VEPs,Pairwise win rate 2.86% | 37.68% | 42.93% | 40.82% | 38.99% |
| CADD_1.7machine learning | 40.32%all other VEPs,Pairwise win rate 2.89% | 38.96% | 41.24% | 40.32% | 41.99% |
| M-CAPmachine learning | 40.05%all other VEPs,Pairwise win rate 7.71% | 38.30% | 41.23% | 40.05% | 39.73% |
| SIFTconventional | 39.27%all other VEPs,Pairwise win rate 1.22% | 39.94% | 38.76% | 40.34% | 41.84% |
| MetaSVMmachine learning | 38.99%all other VEPs,Pairwise win rate 8.13% | 39.40% | 38.54% | 38.99% | 39.91% |
| MVPmachine learning | 36.36%all other VEPs,Pairwise win rate 5.36% | 31.32% | 39.93% | 36.36% | 37.77% |
| CADD_1.6machine learning | 34.99%all other VEPs,Pairwise win rate 1.22% | 34.41% | 35.38% | 35.05% | 36.95% |
| AlphScoremachine learning | 33.46%all other VEPs,Pairwise win rate 11.0% | 32.27% | 34.31% | 33.46% | 34.70% |
| CAPICEmachine learning | 29.43%all other VEPs,Pairwise win rate 3.69% | 29.09% | 29.67% | 29.43% | 30.05% |
| PonP2machine learning | 28.21%all other VEPs,Pairwise win rate 1.22% | 29.98% | 26.93% | 28.85% | 26.66% |
| InMeRFmachine learning | 27.01%all other VEPs,Pairwise win rate 4.11% | 26.03% | 27.77% | 27.01% | 28.33% |
| FathmmXFmachine learning | 24.52%all other VEPs,Pairwise win rate 1.83% | 27.15% | 22.57% | 24.52% | 25.52% |
| LIST-S2specialist model | 22.09%all other VEPs,Pairwise win rate 4.4% | 16.75% | 26.21% | 22.09% | 23.55% |
| sequence_unetspecialist model | 20.27%all other VEPs,Pairwise win rate 1.22% | 20.26% | 20.33% | 20.04% | 20.96% |
| PrimateAImachine learning | 19.93%all other VEPs,Pairwise win rate 1.22% | 17.48% | 21.69% | 19.93% | 20.96% |
| PhD-SNPgmachine learning | 18.92%all other VEPs,Pairwise win rate 0.667% | 20.11% | 18.11% | 18.92% | 20.39% |
| fathmm-MKLmachine learning | 16.14%all other VEPs,Pairwise win rate 0.667% | 15.80% | 16.33% | 16.14% | 17.09% |
| phyloPspecialist model | 15.41%all other VEPs,Pairwise win rate 0.667% | 17.71% | 13.78% | 15.41% | 16.70% |
| BLOSUM62conventional | 14.87%all other VEPs,Pairwise win rate 1.11% | 13.74% | 15.68% | 14.68% | 15.55% |
| FATHMMspecialist model | 14.62%all other VEPs,Pairwise win rate 2.33% | 12.66% | 16.05% | 14.32% | 15.31% |
| LRTspecialist model | 13.58%all other VEPs,Pairwise win rate 1.22% | 15.39% | 12.31% | 13.58% | 14.32% |
| DANNmachine learning | 13.40%all other VEPs,Pairwise win rate 0.667% | 14.51% | 12.62% | 13.40% | 14.51% |
| ProtGPT2foundation model | 11.80%all other VEPs,Pairwise win rate 0% | 8.46% | 14.32% | 11.06% | 11.09% |
| GenoCanyonspecialist model | 9.05%all other VEPs,Pairwise win rate 1.22% | 8.65% | 9.35% | 9.05% | 9.87% |
| GERP++specialist model | 8.80%all other VEPs,Pairwise win rate 0.667% | 9.13% | 8.55% | 8.80% | 9.43% |
| Granthamspecialist model | 8.78%all other VEPs,Pairwise win rate 0% | 7.32% | 9.79% | 9.13% | 10.03% |
| SiPhyspecialist model | 7.48%all other VEPs,Pairwise win rate 0% | 8.13% | 7.01% | 7.48% | 8.32% |
| 3Cnetmachine learning | 7.05%all other VEPs,Pairwise win rate 0% | 3.51% | 9.57% | 6.37% | 6.65% |
| phastConsspecialist model | 7.01%all other VEPs,Pairwise win rate 0.667% | 9.13% | 5.50% | 7.01% | 7.54% |
| fitConsspecialist model | 6.51%all other VEPs,Pairwise win rate 3.12% | 2.48% | 9.52% | 6.51% | 6.66% |
Show all 97 rowsShow the top 10 rows only
Caveats and method
What each column measures:
- DMS benchmark, all 36 DMS sets: Average pairwise win rate of each missense VEP against 96 others, by |Spearman| with 36 human DMS datasets
- DMS benchmark, direct assays only: Average pairwise win rate by |Spearman| with the DMS assays the authors class as direct
- DMS benchmark, indirect assays only: Average pairwise win rate by |Spearman| with the DMS assays the authors class as indirect
- DMS benchmark, SNV-accessible substitutions: Average pairwise win rate by |Spearman| restricted to substitutions reachable by one nucleotide change
- DMS benchmark, SNV-accessible, gaps filled: Average pairwise win rate by |Spearman| on SNV-accessible substitutions with missing predictions set to each VEP's most benign score
- DMS phenotypes are chosen per protein and are not pathogenicity labels.
- Rank scores are relative to the VEP set compared; they are not a correlation and cannot be compared with other tables.
- Human proteins only (36 DMS sets, one per protein).
- Five VEPs were trained on DMS data (Envision, CPT-1, DeMaSk, VARITY_R, VARITY_ER); the authors excluded their results for the proteins used to train them (Table S2), but other methods may have been tuned against DMS data without direct training, for example ESM-1v's model selection (Results, Discussion).
- Per-protein correlations are in the figshare deposit, not retrieved; averages do not predict a single protein.
- Transcribed from the source; no reproduction by Rewire and no human scientific review.
- Which assays are direct is not fully recoverable from Table S1 (four blank assay descriptions).
- Which assays are indirect is not fully recoverable from Table S1.
Protocols: Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS sets, Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays only, Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays only, Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions only, Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filled