Contested result

Nucleotide Transformer and RNAErnie: Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md disagree

Low severity
Not yet reviewed
two-study-conflict
unknown

Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks Nucleotide Transformer above RNAErnie in 7 of 7 comparisons; RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank Nucleotide Transformer above RNAErnie in 7 of their 19 comparisons and RNAErnie higher in 4; 8 are ties. Of those papers, RNAGym fitness leaderboard ranks RNAErnie higher on its own (4 of 4).

Rank Nucleotide Transformer higher (1): Benchmarking pre-trained genomic…
Rank RNAErnie higher (1): RNAGym fitness leaderboard
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Nucleotide Transformer higherTieRNAErnie higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks Nucleotide Transformer higher	Ranks RNAErnie higher	Ties
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and 1 more record · DOI	Agrees	7	0	8
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md · Repository	Agrees	0	4	0

Comparisons in Benchmarking pre-trained genomic… (15)

Protocol	Dataset	Metric	Nucleotide Transformer	RNAErnie	Higher
GTEx tissue-specific splicing prediction, Splice acceptors and donors (You et al. 2026 Supplementary Table 5)	GTEx-derived tissue-specific splice site usage dataset (You et al. 2026)	AUPRC (GTEx splicing task, Splice acceptors and donors, averaged over categories), fraction	0.883	0.806	Nucleotide Transformer
GTEx tissue-specific splicing prediction, Splice-15class (You et al. 2026 Supplementary Table 5)	GTEx-derived tissue-specific splice site usage dataset (You et al. 2026)	AUPRC (GTEx splicing task, Splice-15class, averaged over categories), fraction	0.865	0.687	Nucleotide Transformer
GTEx tissue-specific splicing prediction, Splice-53class (You et al. 2026 Supplementary Table 5)	GTEx-derived tissue-specific splice site usage dataset (You et al. 2026)	AUPRC (GTEx splicing task, Splice-53class, averaged over categories), fraction	0.863	0.685	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Accuracy (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	F1 (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Precision (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Recall (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Accuracy (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	F1 (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Precision (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Recall (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Accuracy (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	F1 (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Precision (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Recall (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie

Comparisons in RNAGym fitness leaderboard (4)

Protocol	Dataset	Metric	Nucleotide Transformer	RNAErnie	Higher
RNAGym v0.1.1 fitness, signed Spearman, Aptamer	RNAGym Aptamer DMS assays (2)	Spearman (mean signed Spearman over the 2 Aptamer assays), unitless	-0.0886	0.0306	RNAErnie
RNAGym v0.1.1 fitness, signed Spearman, macro over 3 ncRNA categories	RNAGym ncRNA DMS assays (31)	Spearman (macro-mean over 3 ncRNA categories (equal category weights)), unitless	0.12	0.194	RNAErnie
RNAGym v0.1.1 fitness, signed Spearman, Ribozyme	RNAGym Ribozyme DMS assays (26)	Spearman (mean signed Spearman over the 26 Ribozyme assays), unitless	0.133	0.134	RNAErnie
RNAGym v0.1.1 fitness, signed Spearman, tRNA	RNAGym tRNA DMS assays (3)	Spearman (mean signed Spearman over the 3 tRNA assays), unitless	0.317	0.416	RNAErnie

What disagrees

studies: 2

Compared: Nucleotide Transformer, RNAErnie
Study that disagrees
Other studies
Rule: two-study-conflict; unknown
Model identity: Matched across studies partly through links not yet reviewed

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-175128b829e1b42e · All contested results · How findings are made and followed up