Contested result

DNABERT and UTR-LM: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks UTR-LM above DNABERT in 23 of 25 comparisons; other studies rank DNABERT above UTR-LM in 8 of 8.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

Benchmarking pre-trained genomic… ranks DNABERT above UTR-LM in all 8 comparisons that are not ties (4 are ties). Zero-shot benchmarking of RNA language… ranks UTR-LM above DNABERT in 23 of its 30 comparisons and DNABERT higher in 2; 5 are ties.

Rank DNABERT higher (1): Benchmarking pre-trained genomic…
Rank UTR-LM higher (1): Zero-shot benchmarking of RNA language… (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

DNABERT higherTieUTR-LM higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks DNABERT higher	Ranks UTR-LM higher	Ties
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning and 1 more record · DOI	Disagrees	2	23	5
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and 1 more record · DOI	Agrees	8	0	4

Comparisons in Zero-shot benchmarking of RNA language… (30)

Protocol	Dataset	Metric	DNABERT	UTR-LM	Higher
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; mean over assays), unitless	0.54	0.551	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; median over assays), unitless	0.535	0.533	Tie
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; mean over assays), unitless	0.064	0.071	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; median over assays), unitless	0.048	0.049	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; mean over assays), unitless	0.077	0.103	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; median over assays), unitless	0.068	0.067	DNABERT
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (mean over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	F1 (attention; mean over structures), unitless	0.017	0.029	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (median over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	F1 (attention; median over structures), unitless	0.01	0.022	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (mean over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	MCC (attention; mean over structures), unitless	0.001	0.016	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (median over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	MCC (attention; median over structures), unitless	0.003	0.006	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (mean over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	F1 (attention; mean over structures), unitless	0.011	0.016	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (median over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	F1 (attention; median over structures), unitless	0.011	0.011	Tie
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (mean over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	MCC (attention; mean over structures), unitless	-0.002	0.006	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (median over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	MCC (attention; median over structures), unitless	0.003	0	DNABERT
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs), unitless	0.585	0.631	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	AUROC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.779	0.801	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	F1 (FFT embedding; mean over random samplings of sequence pairs), unitless	0.721	0.729	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	MCC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.373	0.405	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs), unitless	0.352	0.354	Tie
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	AUROC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.637	0.65	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	F1 (FFT embedding; mean over random samplings of sequence pairs), unitless	0.667	0.669	Tie
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	MCC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.027	0.077	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs), unitless	0.591	0.649	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	AUROC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.786	0.813	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	F1 (CLS embedding; mean over random samplings of sequence pairs), unitless	0.726	0.739	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	MCC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.388	0.445	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs), unitless	0.329	0.34	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	AUROC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.626	0.637	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	F1 (CLS embedding; mean over random samplings of sequence pairs), unitless	0.667	0.667	Tie
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	MCC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.001	0.042	UTR-LM

Comparisons in Benchmarking pre-trained genomic… (12)

Protocol	Dataset	Metric	DNABERT	UTR-LM	Higher
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Accuracy (NT-splice-sites-acceptors, fine-tuned), fraction	0.983	0.969	DNABERT
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	F1 (NT-splice-sites-acceptors, fine-tuned), fraction	0.983	0.969	DNABERT
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Precision (NT-splice-sites-acceptors, fine-tuned), fraction	0.983	0.97	DNABERT
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Recall (NT-splice-sites-acceptors, fine-tuned), fraction	0.983	0.969	DNABERT
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Accuracy (NT-splice-sites-all, fine-tuned), fraction	0.983	0.976	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	F1 (NT-splice-sites-all, fine-tuned), fraction	0.983	0.976	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Precision (NT-splice-sites-all, fine-tuned), fraction	0.983	0.976	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Recall (NT-splice-sites-all, fine-tuned), fraction	0.983	0.976	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Accuracy (NT-splice-sites-donors, fine-tuned), fraction	0.985	0.974	DNABERT
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	F1 (NT-splice-sites-donors, fine-tuned), fraction	0.985	0.974	DNABERT
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Precision (NT-splice-sites-donors, fine-tuned), fraction	0.985	0.974	DNABERT
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Recall (NT-splice-sites-donors, fine-tuned), fraction	0.985	0.974	DNABERT

What disagrees

Other studies: rank DNABERT above UTR-LM in 8 of 8 comparisons
This study: ranks UTR-LM above DNABERT in 23 of 25 comparisons

Compared: DNABERT, UTR-LM
Study that disagrees: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning, Zero-shot benchmarking of RNA language models: Supplementary Information
Other studies: Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications, You et al. 2026, Supplementary Information
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (50)
Study	Tool	Protocol or benchmark	Metric	Value
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.048
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.068
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.585±0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.779±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.721±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.373±0.010
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.637±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.591±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.786±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.726±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.388±0.004
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.329±0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.626±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.540±0.035
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.064±0.059
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.077±0.061
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.017±0.021
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (median over structures)	F1 (attention; median over structures)	0.01
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	0.001±0.024
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.011±0.011
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	-0.002±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.027±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.001±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.551±0.049
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.071±0.068
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.049
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.103±0.097
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.067
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.029±0.032
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (median over structures)	F1 (attention; median over structures)	0.022
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	0.016±0.037
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0.006
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.016±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	0.006±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.631±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.801±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.729±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.405±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.650±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.077±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.649±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.813±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.739±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.445±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.340±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.637±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.042±0.003

In the other studies (16)
Study	Tool	Protocol or benchmark	Metric	Value
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-acceptors, fine-tuned)	0.9693(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-acceptors, fine-tuned)	0.9693(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-acceptors, fine-tuned)	0.9695(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-acceptors, fine-tuned)	0.9693(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-287662856a06602c · All contested results · How findings are made and followed up