Evidence last checked 14 October 2026 · re-checked monthly, next due 14 November 2026
Executive Takeaway
For "Select regulatory variants and genes for functional follow-up", on TraitGym official metrics, Mendelian traits, zero-shot (AUPRC, a proxy protocol), CADD.plus.RawScore scored 0.7122, reported by TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym).
Best reported valueCADD.plus.RawScoreTraitGym official metrics: AUPRC by chromosome, weighted…
AUPRC0.7122TraitGym official metrics, Mendelian traits, zero-shot
Baseline on this protocolNone scoredNo margin is computed
Ranked tools6350 from independent studies
63 tools have a value on this protocol; 50 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.
Ranking
Ranked on TraitGym official metrics, Mendelian traits, zero-shot (AUPRC, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.
14 tools in 63 rows, from 1 source. 50 of 63 results (79%) come from an independent study; 13 are reported by the tool's developers.
Distance to TSS and Activity-by-Contact. Ranking genes by distance to the TSS and the Activity-by-Contact score (element activity times 3D contact) are the established non-learned rules for linking elements to genes. Learned linkers such as ENCODE-rE2G, a supervised model that extends ABC, are judged by how much they improve on them.
Unified model comparison for this use case. Compare tools against protocol baselines in a single view.
Rows marked Contested rest on results that other studies disagree with. The results are still shown. See the disagreements that involve this use case for what disagrees and what we are doing about it.
TraitGym official metrics
Proxy evidenceIndependent study: Reported in a study whose authors did not develop the evaluated tool.Developer-reported: Reported by the developers of the evaluated tool.
AUPRC (higher is better), with the other metrics below each value. Showing the top 10 of 18 rows. The best value in each column is highlighted.
Show all 18 rowsShow the top 10 rows onlyCaveats and method
What each column measures:
Mendelian traits, zero-shot: AUPRC (per-chromosome weighted average, with bootstrap SE) of zero-shot scores from gLMs, sequence-to-function models, CADD and conservation for Mendelian traits causal versus matched noncoding variants
Mendelian traits, linear-probing: AUPRC (per-chromosome weighted average, with bootstrap SE) of linear-probing scores from gLMs, sequence-to-function models, CADD and conservation for Mendelian traits causal versus matched noncoding variants
Complex traits, zero-shot: AUPRC (per-chromosome weighted average, with bootstrap SE) of zero-shot scores from gLMs, sequence-to-function models, CADD and conservation for Complex traits causal versus matched noncoding variants
Complex traits, linear-probing: AUPRC (per-chromosome weighted average, with bootstrap SE) of linear-probing scores from gLMs, sequence-to-function models, CADD and conservation for Complex traits causal versus matched noncoding variants
Benchmark authors developed GPN-MSA and GPN-Promoter (author_reported).
Mendelian positives are curated regulatory pathogenic variants, enriched for conserved promoter and UTR sites, which favours conservation scores and CADD.
Labels are not experimental allele effects.
Official metrics files on a mutable Hugging Face dataset, pinned by revision.
Linear-probing scores are fitted on TraitGym labels with leave-one-chromosome-out training, so they are not comparable with zero-shot scores.
Complex-trait positives come from UK Biobank fine-mapping (high posterior inclusion probability, mostly European ancestry).
Primary benchmark evaluation shown (TraitGym official metrics). 10 additional study protocols (MPRabc K562 CRISPRi enhancer–gene evaluation, Feng eQTL discrimination, Feng ipaQTL discrimination and others) are cited in individual tool drawer rows and under Question, sources and review.
Not covered yet
Automated source review only; independent human scientific review remains outstanding.
No allele-specific endogenous-edit benchmark or equal-budget prospective shortlist benefit established.
Training/test independence unresolved: paper explicitly says benchmarking pipeline does not perform cross-validation.
No locus-to-gene comparison with printed per-method precision and recall was found. CALDERA (Schipper et al. 2026, PLoS Genet) prints only Brier scores for CALDERA, FLAMES, L2G and cS2G (Table 1); its AUPRC comparison is figure-only.
No benchmark of variant-effect predictors against endogenous allele edits (base, prime or saturation editing of noncoding sites) with printed per-method values was found; the linked variant comparisons use reporter assays.
The linked element-gene linking comparisons are by the developers of ENCODE-rE2G and ABC and their co-authors; the astrocyte CRISPRi benchmark (Green et al. 2026, Nat Neurosci), an independent test of ABC and ENCODE-rE2G, reports per-model values only in figures, prose and bootstrap source data.
No linked comparison scores long-context sequence models (Enformer, Borzoi) on reporter allelic effects with a documented input setup; Manzo et al. 2025 describe it two ways, so those rows are excluded.
Gschwind et al. Supplementary Tables 5 and 7 (eQTL and GWAS enrichment benchmarks) and the per-predictor K562 sheets of Table 3 are not extracted.
Manzo et al. causal-SNP top-k results within LD blocks are figure-only.
Select regulatory variants and genes for functional follow-up. Rewire biological model benchmark database, release 2026-11-21-adc15377fab2. https://benchmarks.rewirebio.io/use-cases/regulatory-variant-gene-follow-up/
Release 2026-11-21-adc15377fab2. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.
The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence
Question, sources and review
Who this is for, setting and exclusions
Choose allele edits, regulatory-element perturbations and gene readouts that can distinguish competing explanations for a locus.
Who this is for
Functional geneticists
Researchers interpreting disease-associated loci
Biological setting
Research follow-up of defined disease-associated loci in a specified cell system. Allele-effect prediction, element–gene linking and experimental shortlist selection are separate comparisons.
Research setting
Research only. Molecular effects and regulatory links do not establish clinical variant classification or diagnostic validity.
Outside this use case
Reporter activity does not by itself establish endogenous regulation.
Perturbing an entire regulatory element is different from editing one allele.
A regulatory link alone does not establish a gene’s causal role in disease.
Evidence collection plan
Evidence collection planCollecting evidence
Already on record: 11 comparisons shown · 52 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.
Comparison question
Does a method select more experimentally supported regulatory hypotheses than compatible distance or linking baselines at the same follow-up budget?
Baselines, outcomes and validation requirements
Baselines to include
Distance-based variant/gene or enhancer/gene selection where applicable
ABC, ENCODE-rE2G or scE2G for compatible element–gene linking tasks
Sequence predictors compared only on the allele-effect outputs they produce
Outcomes to measure
Allele-dependent regulatory effects
Endogenous element–gene and allele–gene effects, measured separately
Confirmed follow-up hits per assay budget
Validation requirements
Define the allele-effect, linking or selection endpoint before choosing a comparison.
Hold out loci and studies; audit linkage-disequilibrium and pretraining overlap.
Preserve cell context, genome build, assay and perturbation type.
Close the remaining endpoint, independence and comparator gaps documented in the 30 September 2026 audit and linked article issue; obtain qualified scientific review.
Related articles
Read the background, methods and practical considerations behind this question.
Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.