iGEMME (Livesey and Marsh 2025); Predictions obtained from https://zenodo.org/records/10441521 (Table S2 'Source'); release not stated

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Evaluations and results

6 evaluations · 7 results. Different protocols are not a single leaderboard.

Sorted by Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	Protocol and dataset	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)
iGEMME (Livesey and Marsh 2025)Evaluation	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets, Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	86%	Not reported	Not reported	Not reported	Not reported	Not reported	34.9%
iGEMME (Livesey and Marsh 2025)Evaluation	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense, Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	Not reported	Not reported	Not reported	Not reported	Not reported	70.4%	Not reported
iGEMME (Livesey and Marsh 2025)Evaluation	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only, Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Not reported	89.6%	Not reported	Not reported	Not reported	Not reported	Not reported
iGEMME (Livesey and Marsh 2025)Evaluation	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only, Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Not reported	Not reported	83.4%	Not reported	Not reported	Not reported	Not reported
iGEMME (Livesey and Marsh 2025)Evaluation	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled, Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Not reported	Not reported	Not reported	Not reported	85.8%	Not reported	Not reported
iGEMME (Livesey and Marsh 2025)Evaluation	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only, Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Not reported	Not reported	Not reported	85.9%	Not reported	Not reported	Not reported

All 7 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense Dataset: Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	70.4% Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each) percent · higher Uncertainty: Not reported by the source: Figure 3 error bars (standard error across comparisons) are not printed Coverage: 665/819 proteins with at least 10 ClinVar P/LP and 10 gnomAD missense variants scored by this VEP scored ('proteins' column, cell CJ14)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S8', cell CI14 (row 'iGEMME', column 'rank_score')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	86% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: 36/36 human DMS datasets (one per protein) with predictions for this VEP scored ('proteins' column, cell CV6)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S3', cell CU6 (row 'iGEMME', column 'rank_score')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	34.9% Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S3', cell CW6 (row 'iGEMME', column 'top_5_winrate')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	89.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C9 (row 'iGEMME', column 'rank_score_direct')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	83.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B9 (row 'iGEMME', column 'rank_score_indirect')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	85.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: 36/36 human DMS datasets (one per protein) with predictions for this VEP scored ('proteins' column, cell C6)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S7', cell B6 (row 'iGEMME', column 'rank_score')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	85.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: 36/36 human DMS datasets (one per protein) with predictions for this VEP scored ('proteins' column, cell C6)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S6', cell B6 (row 'iGEMME', column 'rank_score')

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Use this configuration

How it works, versions and access

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

2 source records and release history

Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8) · Original source · 13059_2025_3575_MOESM2_ESM.xlsx as published with PMC12016141
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Original source · PMC12016141 (published 2025-04-22; Genome Biol 26:104)

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: vep-lit-20261014-config-livesey2025-igemme

areas: dna-genomes; proteins-complexes
contexts: research; clinical_research
method types: specialist
reported name: iGEMME
foundation model eligible: false
configuration type: paper_evaluation
version: Predictions obtained from https://zenodo.org/records/10441521 (Table S2 'Source'); release not stated
training overlap: Livesey and Marsh 2025 type: Population-free
access reuse: Livesey and Marsh obtained predictions from: https://zenodo.org/records/10441521
source locator: Supplementary Material 2, Table S2 row 36 ('iGEMME'); printed label 'iGEMME' in Tables S3 and S5-S8
review: reviewer: claude; reviewer note: Claude review agent; no human review claimed. Independent Claude (Opus 5.5) review agent, separate from the extraction agent that built this batch. Automated review only. Transcription, not reproduction.; reviewed at: 2026-10-14T15:30:00Z; artifact sha256: 1b0d792d1a5d3daa95545a7973cabb131c45e42c952e7e5a447a72deef1d1e70; retrieval url: https://static-content.springer.com/esm/art%3A10.1186%2Fs13059-025-03575-w/MediaObjects/13059_2025_3575_MOESM2_ESM.xlsx; method: ai-assisted-source-review; scope-review; method note: Descriptive fields, identity links and missing-metadata reasons checked against the cited sources.; note: Checked; see the dated review for rulings on identity and method type.
provenance note: iGEMME as scored in the Livesey and Marsh 2025 VEP benchmark.

Related records

Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense Dataset: Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	70.4% Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each) percent · higher Uncertainty: Not reported by the source: Figure 3 error bars (standard error across comparisons) are not printed Coverage: 665/819 proteins with at least 10 ClinVar P/LP and 10 gnomAD missense variants scored by this VEP scored ('proteins' column, cell CJ14)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S8', cell CI14 (row 'iGEMME', column 'rank_score')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	86% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: 36/36 human DMS datasets (one per protein) with predictions for this VEP scored ('proteins' column, cell CV6)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S3', cell CU6 (row 'iGEMME', column 'rank_score')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	34.9% Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S3', cell CW6 (row 'iGEMME', column 'top_5_winrate')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	89.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C9 (row 'iGEMME', column 'rank_score_direct')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	83.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B9 (row 'iGEMME', column 'rank_score_indirect')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	85.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: 36/36 human DMS datasets (one per protein) with predictions for this VEP scored ('proteins' column, cell C6)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S7', cell B6 (row 'iGEMME', column 'rank_score')
Configuration: iGEMME (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	85.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: 36/36 human DMS datasets (one per protein) with predictions for this VEP scored ('proteins' column, cell C6)	Independent external evaluation · Source checked Methods, coverage and source iGEMME: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S6', cell B6 (row 'iGEMME', column 'rank_score')