Borzoi and Sei: Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants reverses the order
- Low severity
- Not yet reviewed
- Ordering reversal
- Different tasks across studies
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks Sei above Borzoi in 4 of 4 comparisons; other studies rank Borzoi above Sei in 10 of 10.
An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.
Which studies agree
2 papers rank Borzoi above Sei in all 10 comparisons that are not ties (1 is a tie). Comparative Analysis of Deep Learning… ranks Sei above Borzoi in all 4 of its comparisons.
- Rank Borzoi higher (2): GUANinE v1.1 reveals complementarity…, TraitGym official metrics
- Rank Sei higher (1): Comparative Analysis of Deep Learning… (disagrees)
- Mixed (0): none
Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.
| Paper | Role | Ranks Borzoi higher | Ranks Sei higher | Ties |
|---|---|---|---|---|
| Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants · DOI | Disagrees | 0 | 4 | 0 |
| GUANinE v1.1 reveals complementarity of supervised and genomic language models · DOI | Agrees | 6 | 0 | 1 |
| TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) · Source | Agrees | 4 | 0 | 0 |
Comparisons in Comparative Analysis of Deep Learning… (4)
| Protocol | Dataset | Metric | Borzoi | Sei | Higher |
|---|---|---|---|---|---|
| Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1) | HeLa regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | -0.066 | 0.28 | Sei |
| Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1) | HepG2 regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.125 | 0.298 | Sei |
| Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1) | K562 regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.033 | 0.297 | Sei |
| Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1) | NPC regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.023 | 0.073 | Sei |
Comparisons in GUANinE v1.1 reveals complementarity… (7)
| Protocol | Dataset | Metric | Borzoi | Sei | Higher |
|---|---|---|---|---|---|
| GUANinE v1.1 cadd-snv, Spearman rho x100 (Table 3) | GUANinE cadd-snv test set | Spearman (GUANinE v1.1 test set), unitless | 21 | 19.9 | Borzoi |
| GUANinE v1.1 ccre-propensity, Spearman rho x100 (Table 2) | GUANinE ccre-propensity test set | Spearman (GUANinE v1.1 test set), unitless | 57.7 | 56.4 | Borzoi |
| GUANinE v1.1 cons100, Spearman rho x100 (Table 2) | GUANinE cons100 test set | Spearman (GUANinE v1.1 test set), unitless | 59.7 | 51.6 | Borzoi |
| GUANinE v1.1 cons30, Spearman rho x100 (Table 2) | GUANinE cons30 test set | Spearman (GUANinE v1.1 test set), unitless | 71.5 | 63.2 | Borzoi |
| GUANinE v1.1 dnase-propensity, Spearman rho x100 (Table 2) | GUANinE dnase-propensity test set | Spearman (GUANinE v1.1 test set), unitless | 70.7 | 70.5 | Tie |
| GUANinE v1.1 gpra-c, Spearman rho x100 (Table 2) | GUANinE gpra-c test set | Spearman (GUANinE v1.1 test set), unitless | 71.5 | 68.8 | Borzoi |
| GUANinE v1.1 gpra-d, Spearman rho x100 (Table 2) | GUANinE gpra-d test set | Spearman (GUANinE v1.1 test set), unitless | 75.2 | 72.2 | Borzoi |
Comparisons in TraitGym official metrics (4)
| Protocol | Dataset | Metric | Borzoi | Sei | Higher |
|---|---|---|---|---|---|
| TraitGym Complex traits, linear-probing, AUPRC by chromosome weighted average | TraitGym Complex traits, matched 9 controls | AUPRC (weighted average over chromosomes of per-chromosome AUPRC; subset 'all'), unitless | 0.297 | 0.184 | Borzoi |
| TraitGym Complex traits, zero-shot, AUPRC by chromosome weighted average | TraitGym Complex traits, matched 9 controls | AUPRC (weighted average over chromosomes of per-chromosome AUPRC; subset 'all'), unitless | 0.236 | 0.193 | Borzoi |
| TraitGym Mendelian traits, linear-probing, AUPRC by chromosome weighted average | TraitGym Mendelian traits, matched 9 controls | AUPRC (weighted average over chromosomes of per-chromosome AUPRC; subset 'all'), unitless | 0.493 | 0.349 | Borzoi |
| TraitGym Mendelian traits, zero-shot, AUPRC by chromosome weighted average | TraitGym Mendelian traits, matched 9 controls | AUPRC (weighted average over chromosomes of per-chromosome AUPRC; subset 'all'), unitless | 0.436 | 0.291 | Borzoi |
What disagrees
- Other studies
- rank Borzoi above Sei in 10 of 10 comparisons
- This study
- ranks Sei above Borzoi in 4 of 4 comparisons
- Compared
- Borzoi, Sei
- Study that disagrees
- Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants
- Other studies
- GUANinE v1.1 reveals complementarity of supervised and genomic language models, TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym)
- Rule
- Ordering reversal; different tasks across studies
- Model identity
- Matched across studies partly through links not yet reviewed
The compared numbers
Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.
Follow-up
Current state: Not yet reviewed.
No one has reviewed this finding yet.
How to help resolve this
If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.
Finding discrepancy-81a48372d36fe09e · All contested results · How findings are made and followed up