Borzoi and HyenaDNA: Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants and TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) disagree
- Low severity
- Not yet reviewed
- two-study-conflict
- unknown
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks HyenaDNA above Borzoi in 3 of 4 comparisons; TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) ranks Borzoi above HyenaDNA in 2 of 2 comparisons. No other study gives a verdict, so neither is the outlier.
An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.
Which studies agree
2 papers rank Borzoi above HyenaDNA in 3 of their 6 comparisons and HyenaDNA higher in 3. Of those papers, Comparative Analysis of Deep Learning… ranks HyenaDNA higher on its own (3 of 4).
- Rank Borzoi higher (1): TraitGym official metrics
- Rank HyenaDNA higher (1): Comparative Analysis of Deep Learning…
- Mixed (0): none
Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.
| Paper | Role | Ranks Borzoi higher | Ranks HyenaDNA higher | Ties |
|---|---|---|---|---|
| Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants · DOI | Agrees | 1 | 3 | 0 |
| TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) · Source | Agrees | 2 | 0 | 0 |
Comparisons in Comparative Analysis of Deep Learning… (4)
| Protocol | Dataset | Metric | Borzoi | HyenaDNA | Higher |
|---|---|---|---|---|---|
| Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1) | HeLa regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | -0.066 | 0.055 | HyenaDNA |
| Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1) | HepG2 regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.125 | 0.134 | HyenaDNA |
| Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1) | K562 regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.033 | 0.077 | HyenaDNA |
| Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1) | NPC regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.023 | -0.007 | Borzoi |
Comparisons in TraitGym official metrics (2)
| Protocol | Dataset | Metric | Borzoi | HyenaDNA | Higher |
|---|---|---|---|---|---|
| TraitGym Complex traits, zero-shot, AUPRC by chromosome weighted average | TraitGym Complex traits, matched 9 controls | AUPRC (weighted average over chromosomes of per-chromosome AUPRC; subset 'all'), unitless | 0.236 | 0.111 | Borzoi |
| TraitGym Mendelian traits, zero-shot, AUPRC by chromosome weighted average | TraitGym Mendelian traits, matched 9 controls | AUPRC (weighted average over chromosomes of per-chromosome AUPRC; subset 'all'), unitless | 0.436 | 0.106 | Borzoi |
What disagrees
- studies
- 2
Follow-up
Current state: Not yet reviewed.
No one has reviewed this finding yet.
How to help resolve this
If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.
Finding discrepancy-48140126558a3918 · All contested results · How findings are made and followed up