DNABERT-2 and GENA-LM: GUANinE v1.1 reveals complementarity of supervised and genomic language models reverses the order
- Low severity
- Not yet reviewed
- Ordering reversal
- unknown
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks GENA-LM above DNABERT-2 in 6 of 6 comparisons; 2 of 2 other studies rank DNABERT-2 above GENA-LM.
An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.
Which studies agree
4 papers rank DNABERT-2 above GENA-LM in 62 of their 112 comparisons and GENA-LM higher in 26; 24 are ties. GUANinE v1.1 reveals complementarity… ranks GENA-LM above DNABERT-2 in all 6 of its comparisons.
- Rank DNABERT-2 higher (3): GENEB leaderboard, Benchmarking pre-trained genomic…, Comparative Analysis of Deep Learning…
- Rank GENA-LM higher (1): GUANinE v1.1 reveals complementarity… (disagrees)
- Mixed (1): GenBench
Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.
| Paper | Role | Ranks DNABERT-2 higher | Ranks GENA-LM higher | Ties |
|---|---|---|---|---|
| GUANinE v1.1 reveals complementarity of supervised and genomic language models · DOI | Disagrees | 0 | 6 | 0 |
| GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models · Source | Agrees | 16 | 11 | 24 |
| GENEB leaderboard (leaderboard.json, 13 task-type aggregates) · Repository | Agrees | 31 | 14 | 0 |
| Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and 1 more record · DOI | Agrees | 12 | 0 | 0 |
| Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants · DOI | Agrees | 3 | 1 | 0 |
Comparisons in GUANinE v1.1 reveals complementarity… (6)
| Protocol | Dataset | Metric | DNABERT-2 | GENA-LM | Higher |
|---|---|---|---|---|---|
| GUANinE v1.1 ccre-propensity, Spearman rho x100 (Table 2) | GUANinE ccre-propensity test set | Spearman (GUANinE v1.1 test set), unitless | 32.8 | 34.9 | GENA-LM |
| GUANinE v1.1 cons100, Spearman rho x100 (Table 2) | GUANinE cons100 test set | Spearman (GUANinE v1.1 test set), unitless | 50.4 | 56 | GENA-LM |
| GUANinE v1.1 cons30, Spearman rho x100 (Table 2) | GUANinE cons30 test set | Spearman (GUANinE v1.1 test set), unitless | 60.6 | 66.7 | GENA-LM |
| GUANinE v1.1 dnase-propensity, Spearman rho x100 (Table 2) | GUANinE dnase-propensity test set | Spearman (GUANinE v1.1 test set), unitless | 41.7 | 42.8 | GENA-LM |
| GUANinE v1.1 gpra-c, Spearman rho x100 (Table 2) | GUANinE gpra-c test set | Spearman (GUANinE v1.1 test set), unitless | 69.9 | 71.1 | GENA-LM |
| GUANinE v1.1 gpra-d, Spearman rho x100 (Table 2) | GUANinE gpra-d test set | Spearman (GUANinE v1.1 test set), unitless | 73.9 | 74.8 | GENA-LM |
Comparisons in GenBench (51)
Comparisons in GENEB leaderboard (45)
Comparisons in Benchmarking pre-trained genomic… (12)
Comparisons in Comparative Analysis of Deep Learning… (4)
| Protocol | Dataset | Metric | DNABERT-2 | GENA-LM | Higher |
|---|---|---|---|---|---|
| Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1) | HeLa regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.12 | 0.026 | DNABERT-2 |
| Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1) | HepG2 regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.096 | 0.062 | DNABERT-2 |
| Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1) | K562 regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | 0.086 | 0.077 | DNABERT-2 |
| Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1) | NPC regulatory variant reporter data (Manzo et al. 2025) | Pearson (predicted versus measured log2 fold-change of variant effect), unitless | -0.004 | 0.01 | GENA-LM |
What disagrees
- Other studies
- rank DNABERT-2 above GENA-LM in 15 of 16 comparisons
- This study
- ranks GENA-LM above DNABERT-2 in 6 of 6 comparisons
- Compared
- DNABERT-2, GENA-LM
- Study that disagrees
- GUANinE v1.1 reveals complementarity of supervised and genomic language models
- Other studies
- Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications, You et al. 2026, Supplementary Information, Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants
- Rule
- Ordering reversal; unknown
- Model identity
- Matched across studies partly through links not yet reviewed
The compared numbers
Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.
Follow-up
Current state: Not yet reviewed.
No one has reviewed this finding yet.
How to help resolve this
If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.
Finding discrepancy-b75e4b397013ad74 · All contested results · How findings are made and followed up