rewirebio.iobenchmarks
Contested result

DNABERT-2 and GENA-LM: GUANinE v1.1 reveals complementarity of supervised and genomic language models reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks GENA-LM above DNABERT-2 in 6 of 6 comparisons; 2 of 2 other studies rank DNABERT-2 above GENA-LM.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

4 papers rank DNABERT-2 above GENA-LM in 62 of their 112 comparisons and GENA-LM higher in 26; 24 are ties. GUANinE v1.1 reveals complementarity… ranks GENA-LM above DNABERT-2 in all 6 of its comparisons.

Papers grouped by which system they rank higher: DNABERT-2 or GENA-LMRank DNABERT-2 higher: GENEB leaderboard, Benchmarking pre-trained genomic…, Comparative Analysis of Deep Learning…. Rank GENA-LM higher: GUANinE v1.1 reveals complementarity… (disagrees). Mixed: GenBench.Rank DNABERT-2 higherMixedRank GENA-LM higherGENEB leaderboardGENEB leaderboard (leaderboard.json, 13 task-type aggregates)Benchmarkingpre-trained genomic…Benchmarking pre-trained genomic language models for RNA sequence-related predictive applicationsComparative Analysisof Deep Learning…Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsGenBenchGenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation ModelsGUANinE v1.1 revealscomplementarity…GUANinE v1.1 reveals complementarity of supervised and genomic language models
  • Rank DNABERT-2 higher (3): GENEB leaderboard, Benchmarking pre-trained genomic…, Comparative Analysis of Deep Learning…
  • Rank GENA-LM higher (1): GUANinE v1.1 reveals complementarity… (disagrees)
  • Mixed (1): GenBench

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores DNABERT-2 or GENA-LM higherGUANinE v1.1 reveals complementarity… (disagrees): DNABERT-2 higher in 0, GENA-LM higher in 6, 0 ties. GenBench: DNABERT-2 higher in 16, GENA-LM higher in 11, 24 ties. GENEB leaderboard: DNABERT-2 higher in 31, GENA-LM higher in 14, 0 ties. Benchmarking pre-trained genomic…: DNABERT-2 higher in 12, GENA-LM higher in 0, 0 ties. Comparative Analysis of Deep Learning…: DNABERT-2 higher in 3, GENA-LM higher in 1, 0 ties.Disagrees: GUANinE v1.1 reveals complementarity…GUANinE v1.1 reveals complementarity of supervised and genomic language models6 GENA-LM higher6GenBenchGenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models16 DNABERT-2 higher1624 tie2411 GENA-LM higher11GENEB leaderboardGENEB leaderboard (leaderboard.json, 13 task-type aggregates)31 DNABERT-2 higher3114 GENA-LM higher14Benchmarking pre-trained genomic…Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications12 DNABERT-2 higher12Comparative Analysis of Deep Learning…Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants3 DNABERT-2 higher31 GENA-LM higher1
DNABERT-2 higherTieGENA-LM higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in GUANinE v1.1 reveals complementarity… (6)
Comparisons in GenBench (51)
ProtocolDatasetMetricDNABERT-2GENA-LMHigher
GenBench Bulk RNA expressionGenBench Bulk RNA expressionMSE, unitless0.4510.496DNABERT-2
GenBench Bulk RNA expressionGenBench Bulk RNA expressionSpearman, unitless0.7670.748DNABERT-2
GenBench Drosophila enhancer activity (dev)GenBench Drosophila enhancer activity (dev)Pearson, unitless0.6170.624GENA-LM
GenBench Drosophila enhancer activity (hk)GenBench Drosophila enhancer activity (hk)Pearson, unitless0.7340.74Tie
GenBench Drosophila enhancer activity (Mean)GenBench Drosophila enhancer activity (Mean)Pearson (mean of dev and hk as printed), unitless0.6780.682Tie
GenBench Long-range Promoters PredictionGenBench Long-range Promoters PredictionAccuracy, fraction0.9760.98Tie
GenBench Long-range Species ClassificationGenBench Long-range Species ClassificationAccuracy, fraction0.7470.773GENA-LM
GenBench Long-range splice site prediction (Mean)GenBench Long-range splice site prediction (Mean)AUROC (mean of donor and acceptor as printed), unitless0.6710.679GENA-LM
GenBench Long-range splice site prediction (Splice acceptor)GenBench Long-range splice site prediction (Splice acceptor)AUROC, unitless0.7070.73GENA-LM
GenBench Long-range splice site prediction (Splice donar)GenBench Long-range splice site prediction (Splice donar)AUROC, unitless0.6350.629Tie
GenBench Coding vs IntergenomicGenBench Coding vs IntergenomicAccuracy (task-level value as printed in Table 4), fraction0.9360.932Tie
GenBench Human Core Promoter DetectionGenBench Human Core Promoter DetectionAccuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows), fraction0.8260.814DNABERT-2
GenBench Human Core Promoter Detection, dataset allGenBench Human Core Promoter Detection, dataset allAccuracy, fraction0.8240.816Tie
GenBench Human Core Promoter Detection, dataset notataGenBench Human Core Promoter Detection, dataset notataAccuracy, fraction0.8440.836Tie
GenBench Human Core Promoter Detection, dataset tataGenBench Human Core Promoter Detection, dataset tataAccuracy, fraction0.810.791DNABERT-2
GenBench Human Enhancers CohnGenBench Human Enhancers CohnAccuracy (task-level value as printed in Table 4), fraction0.7590.756Tie
GenBench Human Enhancers EnsemblGenBench Human Enhancers EnsemblAccuracy (task-level value as printed in Table 4), fraction0.9070.911Tie
GenBench Human Ensembl RegulatoryGenBench Human Ensembl RegulatoryAccuracy (task-level value as printed in Table 4), fraction0.8830.881Tie
GenBench Human Nontata PromotersGenBench Human Nontata PromotersAccuracy (task-level value as printed in Table 4), fraction0.9520.966GENA-LM
GenBench Human OCR EnsemblGenBench Human OCR EnsemblAccuracy (task-level value as printed in Table 4), fraction0.7580.79GENA-LM
GenBench Human Promoter DetectionGenBench Human Promoter DetectionAccuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows), fraction0.8990.9Tie
GenBench Human Promoter Detection, dataset allGenBench Human Promoter Detection, dataset allAccuracy, fraction0.9250.929Tie
GenBench Human Promoter Detection, dataset notataGenBench Human Promoter Detection, dataset notataAccuracy, fraction0.9660.962Tie
GenBench Human Promoter Detection, dataset tataGenBench Human Promoter Detection, dataset tataAccuracy, fraction0.8060.81Tie
GenBench Human Splice Site DetectionGenBench Human Splice Site DetectionAccuracy (task-level value as printed in Table 4), fraction0.8810.918GENA-LM
GenBench Human Transcription Factor PredictionGenBench Human Transcription Factor PredictionAccuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows), fraction0.8220.824Tie
GenBench Human Transcription Factor Prediction, dataset 0GenBench Human Transcription Factor Prediction, dataset 0Accuracy, fraction0.8380.832Tie
GenBench Human Transcription Factor Prediction, dataset 1GenBench Human Transcription Factor Prediction, dataset 1Accuracy, fraction0.8520.851Tie
GenBench Human Transcription Factor Prediction, dataset 2GenBench Human Transcription Factor Prediction, dataset 2Accuracy, fraction0.8020.812GENA-LM
GenBench Human Transcription Factor Prediction, dataset 3GenBench Human Transcription Factor Prediction, dataset 3Accuracy, fraction0.7550.777GENA-LM
GenBench Human Transcription Factor Prediction, dataset 4GenBench Human Transcription Factor Prediction, dataset 4Accuracy, fraction0.8620.855Tie
GenBench Human vs WormGenBench Human vs WormAccuracy (task-level value as printed in Table 4), fraction0.9740.97Tie
GenBench Mouse EnhancersGenBench Mouse EnhancersAccuracy (task-level value as printed in Table 4), fraction0.8180.83GENA-LM
GenBench Mouse Transcription Factor PredictionGenBench Mouse Transcription Factor PredictionAccuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows), fraction0.8270.827Tie
GenBench Mouse Transcription Factor Prediction, dataset 0GenBench Mouse Transcription Factor Prediction, dataset 0Accuracy, fraction0.7440.757GENA-LM
GenBench Mouse Transcription Factor Prediction, dataset 1GenBench Mouse Transcription Factor Prediction, dataset 1Accuracy, fraction0.9190.908DNABERT-2
GenBench Mouse Transcription Factor Prediction, dataset 2GenBench Mouse Transcription Factor Prediction, dataset 2Accuracy, fraction0.8970.905Tie
GenBench Mouse Transcription Factor Prediction, dataset 3GenBench Mouse Transcription Factor Prediction, dataset 3Accuracy, fraction0.8420.829DNABERT-2
GenBench Mouse Transcription Factor Prediction, dataset 4GenBench Mouse Transcription Factor Prediction, dataset 4Accuracy, fraction0.7330.733Tie
GenBench Virus Covid Variant ClassificationGenBench Virus Covid Variant ClassificationAccuracy (task-level value as printed in Table 4), fraction0.720.703DNABERT-2
GenBench Yeast Epigenetic Marks PredictionGenBench Yeast Epigenetic Marks PredictionAccuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows), fraction0.8020.783DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H3GenBench Yeast Epigenetic Marks Prediction, dataset H3Accuracy, fraction0.8970.889Tie
GenBench Yeast Epigenetic Marks Prediction, dataset H3K14acGenBench Yeast Epigenetic Marks Prediction, dataset H3K14acAccuracy, fraction0.7790.76DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H3K36me3GenBench Yeast Epigenetic Marks Prediction, dataset H3K36me3Accuracy, fraction0.7970.776DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me1GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me1Accuracy, fraction0.7260.721Tie
GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me2GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me2Accuracy, fraction0.7340.707DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me3GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me3Accuracy, fraction0.760.714DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H3K79me3GenBench Yeast Epigenetic Marks Prediction, dataset H3K79me3Accuracy, fraction0.8410.832DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H3K9acGenBench Yeast Epigenetic Marks Prediction, dataset H3K9acAccuracy, fraction0.8140.784DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H4GenBench Yeast Epigenetic Marks Prediction, dataset H4Accuracy, fraction0.9060.892DNABERT-2
GenBench Yeast Epigenetic Marks Prediction, dataset H4acGenBench Yeast Epigenetic Marks Prediction, dataset H4acAccuracy, fraction0.7690.753DNABERT-2
Comparisons in GENEB leaderboard (45)
ProtocolDatasetMetricDNABERT-2GENA-LMHigher
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Chromatin Acc.'), unitless0.320.499GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Coding/NC'), unitless0.5820.626GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'DNA Methyl.'), unitless0.04140.0625GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Enhancers'), unitless0.3320.272DNABERT-2
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Histone Mod.'), unitless0.2650.181DNABERT-2
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Mouse Enh.'), unitless0.09820.152GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Promoters'), unitless0.4710.49GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Regulatory'), unitless0.2160.151DNABERT-2
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Species Clf.'), unitless0.4020.482GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Splice Sites'), unitless0.2180.158DNABERT-2
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'TF Binding'), unitless0.1470.223GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'Virus/Phage'), unitless0.1340.118DNABERT-2
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; category 'lncRNA'), unitless0.06270.027DNABERT-2
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; macro average: unweighted mean of the 13 category MCCs), unitless0.2530.265GENA-LM
GENEB leaderboard, 10-shot, MCC aggregatesGENEB 100 tasksMCC (10-shot; micro average: mean over all 100 tasks), unitless0.2720.251DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Chromatin Acc.'), unitless0.1560.0442DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Coding/NC'), unitless0.1620.0399DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'DNA Methyl.'), unitless0.002870.00111DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Enhancers'), unitless0.1340.0616DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Histone Mod.'), unitless0.1120.0383DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Mouse Enh.'), unitless0.01880.00967DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Promoters'), unitless0.240.11DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Regulatory'), unitless0.07090.0356DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Species Clf.'), unitless0.1770.117DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Splice Sites'), unitless0.07890.0517DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'TF Binding'), unitless0.06630.0292DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'Virus/Phage'), unitless0.04210.0121DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; category 'lncRNA'), unitless0.01630.0182GENA-LM
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; macro average: unweighted mean of the 13 category MCCs), unitless0.09820.0437DNABERT-2
GENEB leaderboard, 1-shot, MCC aggregatesGENEB 100 tasksMCC (1-shot; micro average: mean over all 100 tasks), unitless0.1190.0526DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Chromatin Acc.'), unitless0.570.461DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Coding/NC'), unitless0.820.83GENA-LM
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'DNA Methyl.'), unitless0.180.198GENA-LM
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Enhancers'), unitless0.4830.477DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Histone Mod.'), unitless0.4690.449DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Mouse Enh.'), unitless0.5270.48DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Promoters'), unitless0.7080.697DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Regulatory'), unitless0.4350.449GENA-LM
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Species Clf.'), unitless0.6250.674GENA-LM
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Splice Sites'), unitless0.4910.42DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'TF Binding'), unitless0.5740.59GENA-LM
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'Virus/Phage'), unitless0.4920.481DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; category 'lncRNA'), unitless0.2650.249DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; macro average: unweighted mean of the 13 category MCCs), unitless0.5110.496DNABERT-2
GENEB leaderboard, full training data, MCC aggregatesGENEB 100 tasksMCC (full training data; micro average: mean over all 100 tasks), unitless0.5060.492DNABERT-2
Comparisons in Benchmarking pre-trained genomic… (12)
ProtocolDatasetMetricDNABERT-2GENA-LMHigher
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, acceptorsAccuracy (NT-splice-sites-acceptors, fine-tuned), fraction0.9780.955DNABERT-2
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, acceptorsF1 (NT-splice-sites-acceptors, fine-tuned), fraction0.9780.954DNABERT-2
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, acceptorsPrecision (NT-splice-sites-acceptors, fine-tuned), fraction0.9780.956DNABERT-2
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, acceptorsRecall (NT-splice-sites-acceptors, fine-tuned), fraction0.9780.954DNABERT-2
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, all sitesAccuracy (NT-splice-sites-all, fine-tuned), fraction0.950.934DNABERT-2
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, all sitesF1 (NT-splice-sites-all, fine-tuned), fraction0.950.934DNABERT-2
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, all sitesPrecision (NT-splice-sites-all, fine-tuned), fraction0.950.934DNABERT-2
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, all sitesRecall (NT-splice-sites-all, fine-tuned), fraction0.950.934DNABERT-2
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, donorsAccuracy (NT-splice-sites-donors, fine-tuned), fraction0.9680.947DNABERT-2
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, donorsF1 (NT-splice-sites-donors, fine-tuned), fraction0.9680.947DNABERT-2
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, donorsPrecision (NT-splice-sites-donors, fine-tuned), fraction0.9680.948DNABERT-2
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Nucleotide Transformer benchmark splice-site dataset, donorsRecall (NT-splice-sites-donors, fine-tuned), fraction0.9680.947DNABERT-2
Comparisons in Comparative Analysis of Deep Learning… (4)
ProtocolDatasetMetricDNABERT-2GENA-LMHigher
Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1)HeLa regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.120.026DNABERT-2
Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1)HepG2 regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.0960.062DNABERT-2
Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1)K562 regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless0.0860.077DNABERT-2
Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1)NPC regulatory variant reporter data (Manzo et al. 2025)Pearson (predicted versus measured log2 fold-change of variant effect), unitless-0.0040.01GENA-LM

What disagrees

Other studies
rank DNABERT-2 above GENA-LM in 15 of 16 comparisons
This study
ranks GENA-LM above DNABERT-2 in 6 of 6 comparisons
Compared
DNABERT-2, GENA-LM
Study that disagrees
GUANinE v1.1 reveals complementarity of supervised and genomic language models
Other studies
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications, You et al. 2026, Supplementary Information, Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants
Rule
Ordering reversal; unknown
Model identity
Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (12)
StudyToolProtocol or benchmarkMetricValue
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsDNABERT-2 (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 ccre-propensity, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)32.8
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsDNABERT-2 (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 cons100, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)50.4
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsDNABERT-2 (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 cons30, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)60.6
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsDNABERT-2 (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 dnase-propensity, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)41.7
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsDNABERT-2 (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 gpra-c, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)69.9
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsDNABERT-2 (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 gpra-d, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)73.9
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsGENA-LM-bert-base-t2t (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 ccre-propensity, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)34.9
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsGENA-LM-bert-base-t2t (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 cons100, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)56
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsGENA-LM-bert-base-t2t (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 cons30, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)66.7
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsGENA-LM-bert-base-t2t (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 dnase-propensity, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)42.8
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsGENA-LM-bert-base-t2t (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 gpra-c, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)71.1
GUANinE v1.1 reveals complementarity of supervised and genomic language modelsGENA-LM-bert-base-t2t (1% eval (evaluated with 1% of the training set), GUANinE v1.1)GUANinE v1.1 gpra-d, Spearman rho x100 (Table 2)Spearman (GUANinE v1.1 test set)74.8
In the other studies (32)
StudyToolProtocol or benchmarkMetricValue
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Accuracy (NT-splice-sites-acceptors, fine-tuned)0.9776(0.0005)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)F1 (NT-splice-sites-acceptors, fine-tuned)0.9776(0.0005)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Precision (NT-splice-sites-acceptors, fine-tuned)0.9777(0.0005)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Recall (NT-splice-sites-acceptors, fine-tuned)0.9776(0.0006)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Accuracy (NT-splice-sites-acceptors, fine-tuned)0.9545(0.0028)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)F1 (NT-splice-sites-acceptors, fine-tuned)0.9544(0.0028)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Precision (NT-splice-sites-acceptors, fine-tuned)0.9555(0.0023)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Recall (NT-splice-sites-acceptors, fine-tuned)0.9543(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Accuracy (NT-splice-sites-all, fine-tuned)0.9499(0.0037)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)F1 (NT-splice-sites-all, fine-tuned)0.9500(0.0036)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Precision (NT-splice-sites-all, fine-tuned)0.9503(0.0034)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Recall (NT-splice-sites-all, fine-tuned)0.9499(0.0037)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Accuracy (NT-splice-sites-all, fine-tuned)0.9336(0.0021)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)F1 (NT-splice-sites-all, fine-tuned)0.9337(0.0023)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Precision (NT-splice-sites-all, fine-tuned)0.9341(0.0026)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Recall (NT-splice-sites-all, fine-tuned)0.9336(0.0021)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Accuracy (NT-splice-sites-donors, fine-tuned)0.9677(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)F1 (NT-splice-sites-donors, fine-tuned)0.9677(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Precision (NT-splice-sites-donors, fine-tuned)0.9678(0.0028)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationDNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Recall (NT-splice-sites-donors, fine-tuned)0.9677(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Accuracy (NT-splice-sites-donors, fine-tuned)0.9472(0.0016)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)F1 (NT-splice-sites-donors, fine-tuned)0.9472(0.0016)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Precision (NT-splice-sites-donors, fine-tuned)0.9477(0.0012)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary InformationGENA-LM-base fine-tuned with an appended convolutional layer (You et al. 2026)NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)Recall (NT-splice-sites-donors, fine-tuned)0.9472(0.0016)
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsDNABERT-2 (Manzo et al. 2025)Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.12
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsGena LM b-multi (Manzo et al. 2025)Allelic reporter effect correlation in HeLa (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.026
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsDNABERT-2 (Manzo et al. 2025)Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.096
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsGena LM b-multi (Manzo et al. 2025)Allelic reporter effect correlation in HepG2 (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.062
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsDNABERT-2 (Manzo et al. 2025)Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.086
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsGena LM b-multi (Manzo et al. 2025)Allelic reporter effect correlation in K562 (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.077
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsDNABERT-2 (Manzo et al. 2025)Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)−0.004
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory VariantsGena LM b-multi (Manzo et al. 2025)Allelic reporter effect correlation in NPC (Manzo et al. 2025 Table 1)Pearson (predicted versus measured log2 fold-change of variant effect)0.01

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-b75e4b397013ad74 · All contested results · How findings are made and followed up