Contested result

Nucleotide Transformer and RNAErnie: RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons; other studies rank Nucleotide Transformer above RNAErnie in 7 of 7.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

Benchmarking pre-trained genomic… ranks Nucleotide Transformer above RNAErnie in all 7 comparisons that are not ties (8 are ties). RNAGym fitness leaderboard ranks RNAErnie above Nucleotide Transformer in all 4 of its comparisons.

Rank Nucleotide Transformer higher (1): Benchmarking pre-trained genomic…
Rank RNAErnie higher (1): RNAGym fitness leaderboard (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Nucleotide Transformer higherTieRNAErnie higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks Nucleotide Transformer higher	Ranks RNAErnie higher	Ties
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md · Repository	Disagrees	0	4	0
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications and 1 more record · DOI	Agrees	7	0	8

Comparisons in RNAGym fitness leaderboard (4)

Protocol	Dataset	Metric	Nucleotide Transformer	RNAErnie	Higher
RNAGym v0.1.1 fitness, signed Spearman, Aptamer	RNAGym Aptamer DMS assays (2)	Spearman (mean signed Spearman over the 2 Aptamer assays), unitless	-0.0886	0.0306	RNAErnie
RNAGym v0.1.1 fitness, signed Spearman, macro over 3 ncRNA categories	RNAGym ncRNA DMS assays (31)	Spearman (macro-mean over 3 ncRNA categories (equal category weights)), unitless	0.12	0.194	RNAErnie
RNAGym v0.1.1 fitness, signed Spearman, Ribozyme	RNAGym Ribozyme DMS assays (26)	Spearman (mean signed Spearman over the 26 Ribozyme assays), unitless	0.133	0.134	RNAErnie
RNAGym v0.1.1 fitness, signed Spearman, tRNA	RNAGym tRNA DMS assays (3)	Spearman (mean signed Spearman over the 3 tRNA assays), unitless	0.317	0.416	RNAErnie

Comparisons in Benchmarking pre-trained genomic… (15)

Protocol	Dataset	Metric	Nucleotide Transformer	RNAErnie	Higher
GTEx tissue-specific splicing prediction, Splice acceptors and donors (You et al. 2026 Supplementary Table 5)	GTEx-derived tissue-specific splice site usage dataset (You et al. 2026)	AUPRC (GTEx splicing task, Splice acceptors and donors, averaged over categories), fraction	0.883	0.806	Nucleotide Transformer
GTEx tissue-specific splicing prediction, Splice-15class (You et al. 2026 Supplementary Table 5)	GTEx-derived tissue-specific splice site usage dataset (You et al. 2026)	AUPRC (GTEx splicing task, Splice-15class, averaged over categories), fraction	0.865	0.687	Nucleotide Transformer
GTEx tissue-specific splicing prediction, Splice-53class (You et al. 2026 Supplementary Table 5)	GTEx-derived tissue-specific splice site usage dataset (You et al. 2026)	AUPRC (GTEx splicing task, Splice-53class, averaged over categories), fraction	0.863	0.685	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Accuracy (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	F1 (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Precision (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, acceptors	Recall (NT-splice-sites-acceptors, fine-tuned), fraction	0.99	0.979	Nucleotide Transformer
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Accuracy (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	F1 (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Precision (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-all splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, all sites	Recall (NT-splice-sites-all, fine-tuned), fraction	0.984	0.981	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Accuracy (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	F1 (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Precision (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie
NT-splice-sites-donors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Nucleotide Transformer benchmark splice-site dataset, donors	Recall (NT-splice-sites-donors, fine-tuned), fraction	0.991	0.983	Tie

What disagrees

Other studies: rank Nucleotide Transformer above RNAErnie in 7 of 7 comparisons
This study: ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons

Compared: Nucleotide Transformer, RNAErnie
Study that disagrees: RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md
Other studies: Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications, You et al. 2026, Supplementary Information
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (8)
Study	Tool	Protocol or benchmark	Metric	Value
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	Nucleotide Transformer (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, Aptamer	Spearman (mean signed Spearman over the 2 Aptamer assays)	-0.0886
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	Nucleotide Transformer (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, macro over 3 ncRNA categories	Spearman (macro-mean over 3 ncRNA categories (equal category weights))	0.12
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	Nucleotide Transformer (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, Ribozyme	Spearman (mean signed Spearman over the 26 Ribozyme assays)	0.133
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	Nucleotide Transformer (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, tRNA	Spearman (mean signed Spearman over the 3 tRNA assays)	0.317
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	RNA-ERNIE (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, Aptamer	Spearman (mean signed Spearman over the 2 Aptamer assays)	0.0306
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	RNA-ERNIE (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, macro over 3 ncRNA categories	Spearman (macro-mean over 3 ncRNA categories (equal category weights))	0.194
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	RNA-ERNIE (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, Ribozyme	Spearman (mean signed Spearman over the 26 Ribozyme assays)	0.134
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md	RNA-ERNIE (RNAGym fitness leaderboard v0.1.1)	RNAGym v0.1.1 fitness, signed Spearman, tRNA	Spearman (mean signed Spearman over the 3 tRNA assays)	0.416

In the other studies (14)
Study	Tool	Protocol or benchmark	Metric	Value
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-acceptors, fine-tuned)	0.9899(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-acceptors, fine-tuned)	0.9899(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-acceptors, fine-tuned)	0.9899(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-acceptors, fine-tuned)	0.9900(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Accuracy (NT-splice-sites-acceptors, fine-tuned)	0.9794(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	F1 (NT-splice-sites-acceptors, fine-tuned)	0.9794(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Precision (NT-splice-sites-acceptors, fine-tuned)	0.9794(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	NT-splice-sites-acceptors splice-site classification, fine-tuned models (You et al. 2026 Supplementary Table 4)	Recall (NT-splice-sites-acceptors, fine-tuned)	0.9794(0.0014)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	GTEx tissue-specific splicing prediction, Splice acceptors and donors (You et al. 2026 Supplementary Table 5)	AUPRC (GTEx splicing task, Splice acceptors and donors, averaged over categories)	0.8826(0.0036)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	GTEx tissue-specific splicing prediction, Splice acceptors and donors (You et al. 2026 Supplementary Table 5)	AUPRC (GTEx splicing task, Splice acceptors and donors, averaged over categories)	0.8063(0.0015)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	GTEx tissue-specific splicing prediction, Splice-15class (You et al. 2026 Supplementary Table 5)	AUPRC (GTEx splicing task, Splice-15class, averaged over categories)	0.8646(0.0032)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	GTEx tissue-specific splicing prediction, Splice-15class (You et al. 2026 Supplementary Table 5)	AUPRC (GTEx splicing task, Splice-15class, averaged over categories)	0.6868(0.0055)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	NucleotideTransformer fine-tuned with an appended convolutional layer (You et al. 2026)	GTEx tissue-specific splicing prediction, Splice-53class (You et al. 2026 Supplementary Table 5)	AUPRC (GTEx splicing task, Splice-53class, averaged over categories)	0.8634(0.0004)
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications; You et al. 2026, Supplementary Information	RNAErnie fine-tuned with an appended convolutional layer (You et al. 2026)	GTEx tissue-specific splicing prediction, Splice-53class (You et al. 2026 Supplementary Table 5)	AUPRC (GTEx splicing task, Splice-53class, averaged over categories)	0.6851(0.0182)

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-3e9818ad9b46ef95 · All contested results · How findings are made and followed up