Contested result

DNABERT-2 and RNA-FM: mRNABench: A curated benchmark for mature mRNA property and function prediction reverses the order

Medium severity
Not yet reviewed
Ordering reversal
Different tasks across studies

mRNABench: A curated benchmark for mature mRNA property and function prediction ranks DNABERT-2 above RNA-FM in 7 of 7 comparisons; other studies rank RNA-FM above DNABERT-2 in 30 of 30.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank RNA-FM above DNABERT-2 in all 30 comparisons that are not ties (4 are ties). MRNABench ranks DNABERT-2 above RNA-FM in all 7 comparisons that are not ties (3 are ties).

Rank RNA-FM higher (2): Zero-shot benchmarking of RNA language…, Benchmarking pre-trained genomic…
Rank DNABERT-2 higher (1): mRNABench (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

RNA-FM higherTieDNABERT-2 higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (3)
Paper	Role	Ranks RNA-FM higher	Ranks DNABERT-2 higher	Ties
mRNABench: A curated benchmark for mature mRNA property and function prediction and 1 more record · DOI	Disagrees	0	7	3
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning and 1 more record · DOI	Agrees	22	0	0
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and 1 more record · DOI	Agrees	8	0	4

Comparisons in mRNABench (10)

Protocol	Dataset	Metric	RNA-FM	DNABERT-2	Higher
mRNABench ECLIP: eCLIP binding site prediction	mRNABench eCLIP (mRNABench split)	AUPRC, percent	35	37.1	DNABERT-2
mRNABench GO: Gene Ontology term prediction	mRNABench GO (mRNABench split)	AUPRC, percent	32.2	32.6	DNABERT-2
mRNABench HL: mRNA half life	mRNABench HL (mRNABench split)	Pearson, unitless	0.47	0.48	DNABERT-2
mRNABench MRL: Mean ribosome load	mRNABench MRL (mRNABench split)	Pearson, unitless	0.29	0.29	Tie
mRNABench MRL-HL-PAIR: Paired mean ribosome load and half life	mRNABench MRL-HL-Pair (mRNABench split)	Pearson, unitless	0.49	0.53	DNABERT-2
mRNABench MRL-MPRA: Mean ribosome load on an MPRA library	mRNABench MRL MPRA (mRNABench split)	Pearson, unitless	0.49	0.57	DNABERT-2
mRNABench MRNA-LOC-LR: mRNA localisation, long range	mRNABench mRNA Loc-LR (mRNABench split)	AUPRC, percent	74.3	76.2	DNABERT-2
mRNABench MRNA-LOC-SR: mRNA localisation, short range	mRNABench mRNA Loc-SR (mRNABench split)	AUPRC, percent	67	67.4	Tie
mRNABench PROT-LOC: Protein localisation	mRNABench Prot Loc (mRNABench split)	AUPRC, percent	32.2	32.5	Tie
mRNABench VEP: Variant effect prediction	mRNABench VEP (mRNABench split)	AUPRC, percent	25.7	26.4	DNABERT-2

Comparisons in Zero-shot benchmarking of RNA language… (22)

Protocol	Dataset	Metric	RNA-FM	DNABERT-2	Higher
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; mean over assays), unitless	0.569	0.543	RNA-FM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; median over assays), unitless	0.55	0.542	RNA-FM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; mean over assays), unitless	0.095	0.069	RNA-FM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; median over assays), unitless	0.075	0.061	RNA-FM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; mean over assays), unitless	0.14	0.085	RNA-FM
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; median over assays), unitless	0.103	0.086	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs), unitless	0.878	0.404	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	AUROC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.953	0.655	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	F1 (FFT embedding; mean over random samplings of sequence pairs), unitless	0.889	0.671	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	MCC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.771	0.132	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs), unitless	0.75	0.369	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	AUROC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.872	0.643	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	F1 (FFT embedding; mean over random samplings of sequence pairs), unitless	0.789	0.668	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	MCC (FFT embedding; mean over random samplings of sequence pairs), unitless	0.568	0.078	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs), unitless	0.891	0.415	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	AUROC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.93	0.673	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	F1 (CLS embedding; mean over random samplings of sequence pairs), unitless	0.898	0.668	RNA-FM
Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	RfamSample (561 Rfam families) (Wang et al. 2026)	MCC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.803	0.106	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs), unitless	0.771	0.379	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	AUROC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.861	0.653	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	F1 (CLS embedding; mean over random samplings of sequence pairs), unitless	0.803	0.667	RNA-FM
Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	ArchiveII-Nr (1273 RNAs) (Wang et al. 2026)	MCC (CLS embedding; mean over random samplings of sequence pairs), unitless	0.635	0.005	RNA-FM

Comparisons in Benchmarking pre-trained genomic… (12)

Protocol	Dataset	Metric	RNA-FM	DNABERT-2	Higher
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Accuracy (NT-splice-sites-acceptors, fine-tuned), fraction	0.98	0.978	Tie
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	F1 (NT-splice-sites-acceptors, fine-tuned), fraction	0.98	0.978	Tie
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Precision (NT-splice-sites-acceptors, fine-tuned), fraction	0.98	0.978	Tie
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Recall (NT-splice-sites-acceptors, fine-tuned), fraction	0.98	0.978	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Accuracy (NT-splice-sites-all, fine-tuned), fraction	0.981	0.95	RNA-FM
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	F1 (NT-splice-sites-all, fine-tuned), fraction	0.981	0.95	RNA-FM
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Precision (NT-splice-sites-all, fine-tuned), fraction	0.981	0.95	RNA-FM
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Recall (NT-splice-sites-all, fine-tuned), fraction	0.981	0.95	RNA-FM
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Accuracy (NT-splice-sites-donors, fine-tuned), fraction	0.984	0.968	RNA-FM
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	F1 (NT-splice-sites-donors, fine-tuned), fraction	0.984	0.968	RNA-FM
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Precision (NT-splice-sites-donors, fine-tuned), fraction	0.984	0.968	RNA-FM
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Recall (NT-splice-sites-donors, fine-tuned), fraction	0.984	0.968	RNA-FM

What disagrees

Other studies: rank RNA-FM above DNABERT-2 in 30 of 30 comparisons
This study: ranks DNABERT-2 above RNA-FM in 7 of 7 comparisons

Compared: DNABERT-2, RNA-FM
Study that disagrees: mRNABench: A curated benchmark for mature mRNA property and function prediction, mRNABench: A curated benchmark for mature mRNA property and function prediction
Other studies: Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications, You et al. 2026, Supplementary Information, Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning, Zero-shot benchmarking of RNA language models: Supplementary Information
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (14)
Study	Tool	Protocol or benchmark	Metric	Value
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench ECLIP: eCLIP binding site prediction	AUPRC	37.1
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench GO: Gene Ontology term prediction	AUPRC	32.6
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench HL: mRNA half life	Pearson	0.48
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench MRL-HL-PAIR: Paired mean ribosome load and half life	Pearson	0.53
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench MRL-MPRA: Mean ribosome load on an MPRA library	Pearson	0.57
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench MRNA-LOC-LR: mRNA localisation, long range	AUPRC	76.2
mRNABench: A curated benchmark for mature mRNA property and function prediction	DNABERT2	mRNABench VEP: Variant effect prediction	AUPRC	26.4
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench ECLIP: eCLIP binding site prediction	AUPRC	35
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench GO: Gene Ontology term prediction	AUPRC	32.2
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench HL: mRNA half life	Pearson	0.47
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench MRL-HL-PAIR: Paired mean ribosome load and half life	Pearson	0.49
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench MRL-MPRA: Mean ribosome load on an MPRA library	Pearson	0.49
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench MRNA-LOC-LR: mRNA localisation, long range	AUPRC	74.3
mRNABench: A curated benchmark for mature mRNA property and function prediction	RNA-FM	mRNABench VEP: Variant effect prediction	AUPRC	25.7

In the other studies (60 results)

In the other studies
Study	Tool	Protocol or benchmark	Metric	Value
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-all, fine-tuned)	0.9499(0.0037)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-all, fine-tuned)	0.9500(0.0036)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-all, fine-tuned)	0.9503(0.0034)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-all, fine-tuned)	0.9499(0.0037)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-all, fine-tuned)	0.9806(0.0008)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-all, fine-tuned)	0.9806(0.0008)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-all, fine-tuned)	0.9806(0.0008)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-all, fine-tuned)	0.9806(0.0008)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-donors, fine-tuned)	0.9677(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-donors, fine-tuned)	0.9677(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-donors, fine-tuned)	0.9678(0.0028)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	DNABERT2 fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-donors, fine-tuned)	0.9677(0.0029)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-donors, fine-tuned)	0.9841(0.0018)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-donors, fine-tuned)	0.9841(0.0018)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-donors, fine-tuned)	0.9842(0.0017)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAFM fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-donors, fine-tuned)	0.9841(0.0019)
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.543±0.031
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.542
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.069±0.052
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.061
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.085±0.056
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.086
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.404±0.003
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.655±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.671±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.132±0.011
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.369±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.643±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.668±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.078±0.018
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.415±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.673±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.668±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.106±0.042
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.379±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.653±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.667±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	DNABERT-2 (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.005±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.569±0.057
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.55
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.095±0.085
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.075
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.140±0.116
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.103
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.878±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.953±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.889±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.771±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (FFT embedding; mean over random samplings of sequence pairs)	0.750±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (FFT embedding; mean over random samplings of sequence pairs)	0.872±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (FFT embedding; mean over random samplings of sequence pairs)	0.789±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), FFT embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (FFT embedding; mean over random samplings of sequence pairs)	0.568±0.002
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.891±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.930±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.898±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot family classification, RfamSample (561 Rfam families), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.803±0.000
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, 1 − OR (mean over repeated random samplings of sequence pairs)	One minus overlap ratio (CLS embedding; mean over random samplings of sequence pairs)	0.771±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, AUC (mean over repeated random samplings of sequence pairs)	AUROC (CLS embedding; mean over random samplings of sequence pairs)	0.861±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, F1 (mean over repeated random samplings of sequence pairs)	F1 (CLS embedding; mean over random samplings of sequence pairs)	0.803±0.001
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	RNA-FM (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot RNA type classification, ArchiveII-Nr (1273 RNAs), CLS embedding, MCC (mean over repeated random samplings of sequence pairs)	MCC (CLS embedding; mean over random samplings of sequence pairs)	0.635±0.001

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-14ca3a59b4201d63 · All contested results · How findings are made and followed up