rewirebio.iobenchmarks
Contested result

ChromBPNet and Borzoi: GUANinE v1.1 reveals complementarity of supervised and genomic language models and Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks Borzoi above ChromBPNet in 5 of 5 comparisons; Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above Borzoi in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

3 papers rank ChromBPNet above Borzoi in 7 of their 21 comparisons and Borzoi higher in 12; 2 are ties. Of those papers, AlphaGenome Nature 2026, GUANinE v1.1 reveals complementarity… rank Borzoi higher on their own (7 of 10, 5 of 7).

Papers grouped by which system they rank higher: ChromBPNet or BorzoiRank ChromBPNet higher: Comparative Analysis of Deep Learning…. Rank Borzoi higher: AlphaGenome Nature 2026, GUANinE v1.1 reveals complementarity…. Mixed: none.Rank ChromBPNet higherMixedRank Borzoi higherComparative Analysisof Deep Learning…Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsAlphaGenome Nature 2026AlphaGenome Nature 2026 supplementary comparison tablesGUANinE v1.1 revealscomplementarity…GUANinE v1.1 reveals complementarity of supervised and genomic language models
  • Rank ChromBPNet higher (1): Comparative Analysis of Deep Learning…
  • Rank Borzoi higher (2): AlphaGenome Nature 2026, GUANinE v1.1 reveals complementarity…
  • Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores ChromBPNet or Borzoi higherAlphaGenome Nature 2026: ChromBPNet higher in 3, Borzoi higher in 7, 0 ties. GUANinE v1.1 reveals complementarity…: ChromBPNet higher in 0, Borzoi higher in 5, 2 ties. Comparative Analysis of Deep Learning…: ChromBPNet higher in 4, Borzoi higher in 0, 0 ties.AlphaGenome Nature 2026AlphaGenome Nature 2026 supplementary comparison tables3 ChromBPNet higher37 Borzoi higher7GUANinE v1.1 reveals complementarity…GUANinE v1.1 reveals complementarity of supervised and genomic language models2 tie25 Borzoi higher5Comparative Analysis of Deep Learning…Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants4 ChromBPNet higher4
ChromBPNet higherTieBorzoi higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in AlphaGenome Nature 2026 (10)
ProtocolDatasetMetricChromBPNetBorzoiHigher
African-ancestry LCL caQTL classification (AlphaGenome paper)African-ancestry LCL caQTL classification: evaluated data subsetAUPRC, unitless0.4150.462Borzoi
European-ancestry LCL caQTL classification (AlphaGenome paper)European-ancestry LCL caQTL classification: evaluated data subsetAUPRC, unitless0.2790.312Borzoi
Yoruba LCL dsQTL classification (AlphaGenome paper)Yoruba LCL dsQTL classification: evaluated data subsetAUPRC, unitless0.5430.606Borzoi
African-ancestry LCL caQTL effect-size prediction (AlphaGenome paper)African-ancestry LCL caQTL effect-size prediction: evaluated data subsetPearson, unitless0.6740.649ChromBPNet
European-ancestry LCL caQTL effect-size prediction (AlphaGenome paper)European-ancestry LCL caQTL effect-size prediction: evaluated data subsetPearson, unitless0.5250.511ChromBPNet
Yoruba LCL dsQTL effect-size prediction (AlphaGenome paper)Yoruba LCL dsQTL effect-size prediction: evaluated data subsetPearson, unitless0.7720.792Borzoi
Microglia caQTL effect-size prediction (AlphaGenome paper)Microglia caQTL effect-size prediction: evaluated data subsetPearson, unitless0.610.616Borzoi
Coronary smooth-muscle caQTL effect-size prediction (AlphaGenome paper)Coronary smooth-muscle caQTL effect-size prediction: evaluated data subsetPearson, unitless0.6580.63ChromBPNet
SPI1 binding QTL classification (AlphaGenome paper)SPI1 binding QTL classification: evaluated data subsetAUPRC, unitless0.3560.468Borzoi
SPI1 binding QTL effect-size prediction (AlphaGenome paper)SPI1 binding QTL effect-size prediction: evaluated data subsetPearson, unitless0.520.535Borzoi
Comparisons in GUANinE v1.1 reveals complementarity… (7)
Comparisons in Comparative Analysis of Deep Learning… (4)
ProtocolDatasetMetricChromBPNetBorzoiHigher
Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1)HeLa regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.289-0.066ChromBPNet
Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1)HepG2 regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.2950.125ChromBPNet
Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1)K562 regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.2870.033ChromBPNet
Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1)NPC regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.080.023ChromBPNet

What disagrees

studies
2
Compared
ChromBPNet, Borzoi
Study that disagrees
Other studies
Rule
Two-study conflict; unknown
Model identity
Matched across studies partly through links not yet reviewed

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-ee4e9764074d9a0b · All contested results · How findings are made and followed up