Contested result

Evo and LucaOne: GENEB leaderboard (leaderboard.json, 13 task-type aggregates) reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

GENEB leaderboard (leaderboard.json, 13 task-type aggregates) ranks LucaOne above Evo in 40 of 45 comparisons; other studies rank Evo above LucaOne in 6 of 6.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

Zero-shot benchmarking of RNA language… ranks Evo above LucaOne in all 6 of its comparisons. GENEB leaderboard ranks LucaOne above Evo in 40 of its 45 comparisons and Evo higher in 5.

Rank Evo higher (1): Zero-shot benchmarking of RNA language…
Rank LucaOne higher (1): GENEB leaderboard (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Evo higherTieLucaOne higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks Evo higher	Ranks LucaOne higher	Ties
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) · Repository	Disagrees	5	40	0
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning and 1 more record · DOI	Agrees	6	0	0

Comparisons in GENEB leaderboard (45)

Protocol	Dataset	Metric	Evo	LucaOne	Higher
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Chromatin Acc.'), unitless	0.289	0.478	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Coding/NC'), unitless	0.539	0.746	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'DNA Methyl.'), unitless	0.0458	0.0665	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Enhancers'), unitless	0.288	0.304	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Histone Mod.'), unitless	0.258	0.25	Evo
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Mouse Enh.'), unitless	0.12	0.257	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Promoters'), unitless	0.436	0.538	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Regulatory'), unitless	0.125	0.183	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Species Clf.'), unitless	0.207	0.391	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Splice Sites'), unitless	0.0657	0.298	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'TF Binding'), unitless	0.0386	0.254	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Virus/Phage'), unitless	0.0941	0.267	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'lncRNA'), unitless	0.0189	0.178	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs), unitless	0.194	0.324	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; micro average: mean over all 100 tasks), unitless	0.232	0.313	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Chromatin Acc.'), unitless	0.14	0.15	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Coding/NC'), unitless	0.293	0.375	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'DNA Methyl.'), unitless	0.0208	0.0148	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Enhancers'), unitless	0.165	0.16	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Histone Mod.'), unitless	0.108	0.0957	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Mouse Enh.'), unitless	0.0568	0.0698	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Promoters'), unitless	0.279	0.287	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Regulatory'), unitless	0.0788	0.0352	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Species Clf.'), unitless	0.103	0.176	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Splice Sites'), unitless	0.0643	0.0774	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'TF Binding'), unitless	0.0084	0.092	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Virus/Phage'), unitless	-0.00343	0.0798	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'lncRNA'), unitless	0.00964	0.0554	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs), unitless	0.102	0.128	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; micro average: mean over all 100 tasks), unitless	0.126	0.136	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Chromatin Acc.'), unitless	0.284	0.662	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Coding/NC'), unitless	0.719	0.901	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'DNA Methyl.'), unitless	0.0732	0.323	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Enhancers'), unitless	0.37	0.479	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Histone Mod.'), unitless	0.397	0.533	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Mouse Enh.'), unitless	0.287	0.597	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Promoters'), unitless	0.537	0.746	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Regulatory'), unitless	0.269	0.492	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Species Clf.'), unitless	0.285	0.662	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Splice Sites'), unitless	0.16	0.636	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'TF Binding'), unitless	0.173	0.605	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Virus/Phage'), unitless	0.205	0.519	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'lncRNA'), unitless	0.119	0.508	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; macro average: unweighted mean of the 13 category MCCs), unitless	0.298	0.589	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; micro average: mean over all 100 tasks), unitless	0.342	0.579	LucaOne

Comparisons in Zero-shot benchmarking of RNA language… (6)

Protocol	Dataset	Metric	Evo	LucaOne	Higher
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; mean over assays), unitless	0.588	0.559	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; median over assays), unitless	0.586	0.544	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; mean over assays), unitless	0.132	0.088	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; median over assays), unitless	0.13	0.066	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; mean over assays), unitless	0.179	0.118	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; median over assays), unitless	0.178	0.091	Evo

What disagrees

Other studies: rank Evo above LucaOne in 6 of 6 comparisons
This study: ranks LucaOne above Evo in 40 of 45 comparisons

Compared: Evo, LucaOne
Study that disagrees: GENEB leaderboard (leaderboard.json, 13 task-type aggregates)
Other studies: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning, Zero-shot benchmarking of RNA language models: Supplementary Information
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (90)
Study	Tool	Protocol or benchmark	Metric	Value
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Chromatin Acc.')	0.289
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Coding/NC')	0.539
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'DNA Methyl.')	0.0458
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Enhancers')	0.288
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Histone Mod.')	0.258
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'lncRNA')	0.0189
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs)	0.194
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; micro average: mean over all 100 tasks)	0.232
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Mouse Enh.')	0.12
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Promoters')	0.436
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Regulatory')	0.125
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Species Clf.')	0.207
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Splice Sites')	0.0657
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'TF Binding')	0.0386
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Virus/Phage')	0.0941
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Chromatin Acc.')	0.14
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Coding/NC')	0.293
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'DNA Methyl.')	0.0208
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Enhancers')	0.165
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Histone Mod.')	0.108
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'lncRNA')	0.00964
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs)	0.102
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; micro average: mean over all 100 tasks)	0.126
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Mouse Enh.')	0.0568
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Promoters')	0.279
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Regulatory')	0.0788
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Species Clf.')	0.103
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Splice Sites')	0.0643
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'TF Binding')	0.0084
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Virus/Phage')	-0.00343
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Chromatin Acc.')	0.284
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Coding/NC')	0.719
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'DNA Methyl.')	0.0732
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Enhancers')	0.37
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Histone Mod.')	0.397
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'lncRNA')	0.119
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; macro average: unweighted mean of the 13 category MCCs)	0.298
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; micro average: mean over all 100 tasks)	0.342
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Mouse Enh.')	0.287
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Promoters')	0.537
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Regulatory')	0.269
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Species Clf.')	0.285
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Splice Sites')	0.16
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'TF Binding')	0.173
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Virus/Phage')	0.205
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Chromatin Acc.')	0.478
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Coding/NC')	0.746
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'DNA Methyl.')	0.0665
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Enhancers')	0.304
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Histone Mod.')	0.25
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'lncRNA')	0.178
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs)	0.324
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; micro average: mean over all 100 tasks)	0.313
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Mouse Enh.')	0.257
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Promoters')	0.538
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Regulatory')	0.183
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Species Clf.')	0.391
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Splice Sites')	0.298
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'TF Binding')	0.254
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Virus/Phage')	0.267
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Chromatin Acc.')	0.15
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Coding/NC')	0.375
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'DNA Methyl.')	0.0148
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Enhancers')	0.16
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Histone Mod.')	0.0957
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'lncRNA')	0.0554
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs)	0.128
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; micro average: mean over all 100 tasks)	0.136
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Mouse Enh.')	0.0698
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Promoters')	0.287
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Regulatory')	0.0352
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Species Clf.')	0.176
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Splice Sites')	0.0774
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'TF Binding')	0.092
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Virus/Phage')	0.0798
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Chromatin Acc.')	0.662
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Coding/NC')	0.901
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'DNA Methyl.')	0.323
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Enhancers')	0.479
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Histone Mod.')	0.533
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'lncRNA')	0.508
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; macro average: unweighted mean of the 13 category MCCs)	0.589
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; micro average: mean over all 100 tasks)	0.579
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Mouse Enh.')	0.597
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Promoters')	0.746
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Regulatory')	0.492
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Species Clf.')	0.662
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Splice Sites')	0.636
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'TF Binding')	0.605
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Virus/Phage')	0.519

In the other studies (12)
Study	Tool	Protocol or benchmark	Metric	Value
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.588±0.056
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.586
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.132±0.088
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.13
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.179±0.114
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.178
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.559±0.055
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.088±0.082
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.066
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.118±0.105
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.091
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (G36.8M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.544

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-827a6ea13a9e5fa3 · All contested results · How findings are made and followed up