Contested result

DNABERT and UTR-LM: Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks DNABERT above UTR-LM in 8 of 8 comparisons; other studies rank UTR-LM above DNABERT in 23 of 25.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

Zero-shot benchmarking of RNA language… ranks UTR-LM above DNABERT in 23 of its 30 comparisons and DNABERT higher in 2; 5 are ties. Benchmarking pre-trained genomic… ranks DNABERT above UTR-LM in all 8 comparisons that are not ties (4 are ties).

Rank UTR-LM higher (1): Zero-shot benchmarking of RNA language…
Rank DNABERT higher (1): Benchmarking pre-trained genomic… (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

UTR-LM higherTieDNABERT higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks UTR-LM higher	Ranks DNABERT higher	Ties
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and 1 more record · DOI	Disagrees	0	8	4
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning and 1 more record · DOI	Agrees	23	2	5

Comparisons in Benchmarking pre-trained genomic… (12)

Protocol	Dataset	Metric	UTR-LM	DNABERT	Higher
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Accuracy (NT-splice-sites-acceptors, fine-tuned), fraction	0.969	0.983	DNABERT
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	F1 (NT-splice-sites-acceptors, fine-tuned), fraction	0.969	0.983	DNABERT
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Precision (NT-splice-sites-acceptors, fine-tuned), fraction	0.97	0.983	DNABERT
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Recall (NT-splice-sites-acceptors, fine-tuned), fraction	0.969	0.983	DNABERT
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Accuracy (NT-splice-sites-all, fine-tuned), fraction	0.976	0.983	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	F1 (NT-splice-sites-all, fine-tuned), fraction	0.976	0.983	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Precision (NT-splice-sites-all, fine-tuned), fraction	0.976	0.983	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Recall (NT-splice-sites-all, fine-tuned), fraction	0.976	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Accuracy (NT-splice-sites-donors, fine-tuned), fraction	0.974	0.985	DNABERT
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	F1 (NT-splice-sites-donors, fine-tuned), fraction	0.974	0.985	DNABERT
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Precision (NT-splice-sites-donors, fine-tuned), fraction	0.974	0.985	DNABERT
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Recall (NT-splice-sites-donors, fine-tuned), fraction	0.974	0.985	DNABERT

Comparisons in Zero-shot benchmarking of RNA language… (30)

Protocol	Dataset	Metric	UTR-LM	DNABERT	Higher
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; mean over assays), unitless	0.551	0.54	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; median over assays), unitless	0.533	0.535	Tie
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; mean over assays), unitless	0.071	0.064	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; median over assays), unitless	0.049	0.048	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; mean over assays), unitless	0.103	0.077	UTR-LM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; median over assays), unitless	0.067	0.068	DNABERT
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (mean over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	F1 (attention; mean over structures), unitless	0.029	0.017	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (median over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	F1 (attention; median over structures), unitless	0.022	0.01	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (mean over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	MCC (attention; mean over structures), unitless	0.016	0.001	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (median over structures)	TS test set (70 PDB structures) (Wang et al. 2026)	MCC (attention; median over structures), unitless	0.006	0.003	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (mean over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	F1 (attention; mean over structures), unitless	0.016	0.011	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (median over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	F1 (attention; median over structures), unitless	0.011	0.011	Tie
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (mean over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	MCC (attention; mean over structures), unitless	0.006	-0.002	UTR-LM
Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (median over structures)	TS-Hard test set (15 PDB structures) (Wang et al. 2026)	MCC (attention; median over structures), unitless	0	0.003	DNABERT
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs), unitless	0.631	0.585	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	AUROC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.801	0.779	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	F1 (FFT embedding; mean over random samplings of sequence pairs), unitless	0.729	0.721	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	MCC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.405	0.373	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs), unitless	0.354	0.352	Tie
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	AUROC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.65	0.637	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	F1 (FFT embedding; mean over random samplings of sequence pairs), unitless	0.669	0.667	Tie
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	MCC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.077	0.027	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs), unitless	0.649	0.591	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	AUROC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.813	0.786	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	F1 (CLS embedding; mean over random samplings of sequence pairs), unitless	0.739	0.726	UTR-LM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	MCC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.445	0.388	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs), unitless	0.34	0.329	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	AUROC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.637	0.626	UTR-LM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	F1 (CLS embedding; mean over random samplings of sequence pairs), unitless	0.667	0.667	Tie
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	MCC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.042	0.001	UTR-LM

What disagrees

Other studies: rank UTR-LM above DNABERT in 23 of 25 comparisons
This study: ranks DNABERT above UTR-LM in 8 of 8 comparisons

Compared: DNABERT, UTR-LM
Study that disagrees: Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications, You et al. 2026, Supplementary Information
Other studies: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning, Zero-shot benchmarking of RNA language models: Supplementary Information
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (16)
Study	Tool	Protocol or benchmark	Metric	Value
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-acceptors, fine-tuned)	0.9832(0.0011)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-acceptors, fine-tuned)	0.9693(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-acceptors, fine-tuned)	0.9693(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-acceptors, fine-tuned)	0.9695(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-acceptors, fine-tuned)	0.9693(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-donors, fine-tuned)	0.9851(0.0007)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	UTRLM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-donors, fine-tuned)	0.9741(0.0000)

In the other studies (50 results)

In the other studies
Study	Tool	Protocol or benchmark	Metric	Value
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.048
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.068
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.585±0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.779±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.721±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.373±0.010
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.637±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.591±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.786±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.726±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.388±0.004
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.329±0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=3) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.626±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.540±0.035
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.064±0.059
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.077±0.061
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.017±0.021
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (median over structures)	F1 (attention; median over structures)	0.01
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	0.001±0.024
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.011±0.011
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	-0.002±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.027±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT (k=6) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.001±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.551±0.049
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.071±0.068
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.049
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.103±0.097
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.067
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.029±0.032
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, F1 score (median over structures)	F1 (attention; median over structures)	0.022
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	0.016±0.037
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS test set (70 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0.006
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, F1 score (mean over structures)	F1 (attention; mean over structures)	0.016±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (mean over structures)	MCC (attention; mean over structures)	0.006±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot secondary structure, TS-Hard test set (15 PDB structures), attention, MCC (median over structures)	MCC (attention; median over structures)	0
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.631±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.801±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.729±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.405±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.650±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.077±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.649±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.813±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.739±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.445±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.340±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.637±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	UTR_LM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.042±0.003

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-e5753219592fcfce · All contested results · How findings are made and followed up