rewirebio.iobenchmarks
Contested result

Evo 2 and CADD: Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) and evo2-cloud-operations recorded run b75c67aa: run receipt, Google Cloud H100 (P2, attempt-1) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks CADD above Evo 2 in 36 of 47 comparisons; evo2-cloud-operations recorded run b75c67aa: run receipt, Google Cloud H100 (P2, attempt-1) ranks Evo 2 above CADD in 3 of 3 comparisons. No other study gives a verdict, so neither is the outlier.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank Evo 2 above CADD in 14 of their 56 comparisons and CADD higher in 36; 6 are ties. Of those papers, Genome modelling and design across all… ranks CADD higher on its own (36 of 52).

Papers grouped by which system they rank higher: Evo 2 or CADDRank Evo 2 higher: evo2-cloud-operations recorded run…. Rank CADD higher: Genome modelling and design across all…. Mixed: none.Rank Evo 2 higherMixedRank CADD higherevo2-cloud-operationsrecorded run…evo2-cloud-operations recorded run b75c67aa: run receipt, Google Cloud H100 (P2, attempt-1)Genome modelling anddesign across all…Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026)
  • Rank Evo 2 higher (1): evo2-cloud-operations recorded run…
  • Rank CADD higher (1): Genome modelling and design across all…
  • Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores Evo 2 or CADD higherGenome modelling and design across all…: Evo 2 higher in 11, CADD higher in 36, 5 ties. evo2-cloud-operations recorded run…: Evo 2 higher in 3, CADD higher in 0, 1 tie.Genome modelling and design across all…Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026)11 Evo 2 higher115 tie536 CADD higher36evo2-cloud-operations recorded run…evo2-cloud-operations recorded run b75c67aa: run receipt, Google Cloud H100 (P2, attempt-1)3 Evo 2 higher31 tie1
Evo 2 higherTieCADD higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in Genome modelling and design across all… (52)
ProtocolDatasetMetricEvo 2CADDHigher
Zero-shot BRCA1 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUPRC (zero-shot; BRCA1 'Both' (coding and noncoding) SNVs), unitless0.6770.73CADD
Zero-shot BRCA1 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUROC (zero-shot; BRCA1 'Both' (coding and noncoding) SNVs), unitless0.9010.876Evo 2
Zero-shot BRCA1 coding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUPRC (zero-shot; BRCA1 coding SNVs), unitless0.5720.521Evo 2
Zero-shot BRCA1 coding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUROC (zero-shot; BRCA1 coding SNVs), unitless0.8430.814Evo 2
Zero-shot BRCA1 intronic SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUPRC (zero-shot; BRCA1 intronic SNVs), unitless0.4240.362Evo 2
Zero-shot BRCA1 intronic SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUROC (zero-shot; BRCA1 intronic SNVs), unitless0.8870.813Evo 2
Zero-shot BRCA1 noncoding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUPRC (zero-shot; BRCA1 noncoding SNVs), unitless0.9030.8Evo 2
Zero-shot BRCA1 noncoding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUROC (zero-shot; BRCA1 noncoding SNVs), unitless0.9740.909Evo 2
Zero-shot BRCA1 splice region SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUPRC (zero-shot; BRCA1 splice region SNVs), unitless0.9180.856Evo 2
Zero-shot BRCA1 splice region SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA1 saturation genome editing SNVs, Findlay et al. 2018 classes (Brixi et al. 2026)AUROC (zero-shot; BRCA1 splice region SNVs), unitless0.9590.89Evo 2
Zero-shot BRCA2 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA2 SNVs with functional classifications, Huang et al. 2025 (Brixi et al. 2026)AUPRC (zero-shot; BRCA2 'Both' (coding and noncoding) SNVs), unitless0.5330.739CADD
Zero-shot BRCA2 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA2 SNVs with functional classifications, Huang et al. 2025 (Brixi et al. 2026)AUROC (zero-shot; BRCA2 'Both' (coding and noncoding) SNVs), unitless0.8450.884CADD
Zero-shot BRCA2 coding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA2 SNVs with functional classifications, Huang et al. 2025 (Brixi et al. 2026)AUPRC (zero-shot; BRCA2 coding SNVs), unitless0.3330.402CADD
Zero-shot BRCA2 coding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA2 SNVs with functional classifications, Huang et al. 2025 (Brixi et al. 2026)AUROC (zero-shot; BRCA2 coding SNVs), unitless0.7680.787CADD
Zero-shot BRCA2 noncoding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA2 SNVs with functional classifications, Huang et al. 2025 (Brixi et al. 2026)AUPRC (zero-shot; BRCA2 noncoding SNVs), unitless0.7860.867CADD
Zero-shot BRCA2 noncoding SNVs classification (Evo 2 paper, Supplementary Data 1)BRCA2 SNVs with functional classifications, Huang et al. 2025 (Brixi et al. 2026)AUROC (zero-shot; BRCA2 noncoding SNVs), unitless0.910.924CADD
Zero-shot ClinVar 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar 'Both' (coding and noncoding) SNVs), unitless0.950.984CADD
Zero-shot ClinVar 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar 'Both' (coding and noncoding) SNVs), unitless0.970.988CADD
Zero-shot ClinVar coding SNVs classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs), unitless0.8890.952CADD
Zero-shot ClinVar coding SNVs classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs), unitless0.8410.927CADD
Zero-shot ClinVar coding SNVs, far from splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs, far from splice site), unitless0.8850.949CADD
Zero-shot ClinVar coding SNVs, far from splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs, far from splice site), unitless0.8410.925CADD
Zero-shot ClinVar coding SNVs, highly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs, highly conserved), unitless0.9180.961CADD
Zero-shot ClinVar coding SNVs, highly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs, highly conserved), unitless0.7760.883CADD
Zero-shot ClinVar coding SNVs, moderately conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs, moderately conserved), unitless0.6670.818CADD
Zero-shot ClinVar coding SNVs, moderately conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs, moderately conserved), unitless0.830.905CADD
Zero-shot ClinVar coding SNVs, near splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs, near splice site), unitless0.9310.977CADD
Zero-shot ClinVar coding SNVs, near splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs, near splice site), unitless0.8270.934CADD
Zero-shot ClinVar coding SNVs, not conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs, not conserved), unitless0.520.623CADD
Zero-shot ClinVar coding SNVs, not conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs, not conserved), unitless0.8680.88CADD
Zero-shot ClinVar coding SNVs, poorly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar coding SNVs, poorly conserved), unitless0.6020.759CADD
Zero-shot ClinVar coding SNVs, poorly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar coding SNVs, poorly conserved), unitless0.860.912CADD
Zero-shot ClinVar low SpliceAI and Pangolin scores (both below 0.1) SNVs classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar low SpliceAI and Pangolin scores (both below 0.1) SNVs), unitless0.3110.541CADD
Zero-shot ClinVar low SpliceAI and Pangolin scores (both below 0.1) SNVs classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar low SpliceAI and Pangolin scores (both below 0.1) SNVs), unitless0.920.92Tie
Zero-shot ClinVar noncoding SNVs, far from splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, far from splice site), unitless0.3430.506CADD
Zero-shot ClinVar noncoding SNVs, far from splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, far from splice site), unitless0.8550.873CADD
Zero-shot ClinVar noncoding SNVs, highly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, highly conserved), unitless0.9940.998Tie
Zero-shot ClinVar noncoding SNVs, highly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, highly conserved), unitless0.9650.983CADD
Zero-shot ClinVar noncoding SNVs, intergenic classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, intergenic), unitless0.7750.848CADD
Zero-shot ClinVar noncoding SNVs, intergenic classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, intergenic), unitless0.8740.913CADD
Zero-shot ClinVar noncoding SNVs, moderately conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, moderately conserved), unitless0.7920.79Tie
Zero-shot ClinVar noncoding SNVs, moderately conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, moderately conserved), unitless0.9410.923Evo 2
Zero-shot ClinVar noncoding SNVs, near splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, near splice site), unitless0.9970.998Tie
Zero-shot ClinVar noncoding SNVs, near splice site classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, near splice site), unitless0.9880.991Tie
Zero-shot ClinVar noncoding SNVs, not conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, not conserved), unitless0.3560.489CADD
Zero-shot ClinVar noncoding SNVs, not conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, not conserved), unitless0.8480.834Evo 2
Zero-shot ClinVar noncoding SNVs, poorly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUPRC (zero-shot; ClinVar noncoding SNVs, poorly conserved), unitless0.5540.607CADD
Zero-shot ClinVar noncoding SNVs, poorly conserved classification (Evo 2 paper, Supplementary Data 1)ClinVar October 2024 release, filtered (Brixi et al. 2026)AUROC (zero-shot; ClinVar noncoding SNVs, poorly conserved), unitless0.8970.908CADD
Zero-shot SpliceVarDB 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)SpliceVarDB experimentally validated splice variants (Brixi et al. 2026)AUPRC (zero-shot; SpliceVarDB 'Both' (coding and noncoding) SNVs), unitless0.850.874CADD
Zero-shot SpliceVarDB 'Both' (coding and noncoding) SNVs classification (Evo 2 paper, Supplementary Data 1)SpliceVarDB experimentally validated splice variants (Brixi et al. 2026)AUROC (zero-shot; SpliceVarDB 'Both' (coding and noncoding) SNVs), unitless0.8150.832CADD
Zero-shot SpliceVarDB exonic SNVs classification (Evo 2 paper, Supplementary Data 1)SpliceVarDB experimentally validated splice variants (Brixi et al. 2026)AUPRC (zero-shot; SpliceVarDB exonic SNVs), unitless0.5230.784CADD
Zero-shot SpliceVarDB exonic SNVs classification (Evo 2 paper, Supplementary Data 1)SpliceVarDB experimentally validated splice variants (Brixi et al. 2026)AUROC (zero-shot; SpliceVarDB exonic SNVs), unitless0.6840.799CADD
Comparisons in evo2-cloud-operations recorded run… (4)

What disagrees

studies
2
Compared
Evo 2, CADD
Study that disagrees
Other studies
Rule
Two-study conflict; different-task
Model identity
Matched across studies partly through links not yet reviewed

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-78d1aa1f01f5863d · All contested results · How findings are made and followed up