Contested result

Evo and LucaOne: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above LucaOne in 6 of 6 comparisons; other studies rank LucaOne above Evo in 40 of 45.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

GENEB leaderboard ranks LucaOne above Evo in 40 of its 45 comparisons and Evo higher in 5. Zero-shot benchmarking of RNA language… ranks Evo above LucaOne in all 6 of its comparisons.

Rank LucaOne higher (1): GENEB leaderboard
Rank Evo higher (1): Zero-shot benchmarking of RNA language… (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

LucaOne higherTieEvo higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks LucaOne higher	Ranks Evo higher	Ties
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning and 1 more record · DOI	Disagrees	0	6	0
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) · Repository	Agrees	40	5	0

Comparisons in Zero-shot benchmarking of RNA language… (6)

Protocol	Dataset	Metric	LucaOne	Evo	Higher
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; mean over assays), unitless	0.559	0.588	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; median over assays), unitless	0.544	0.586	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; mean over assays), unitless	0.088	0.132	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; median over assays), unitless	0.066	0.13	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; mean over assays), unitless	0.118	0.179	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; median over assays), unitless	0.091	0.178	Evo

Comparisons in GENEB leaderboard (45)

Protocol	Dataset	Metric	LucaOne	Evo	Higher
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Chromatin Acc.'), unitless	0.478	0.289	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Coding/NC'), unitless	0.746	0.539	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'DNA Methyl.'), unitless	0.0665	0.0458	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Enhancers'), unitless	0.304	0.288	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Histone Mod.'), unitless	0.25	0.258	Evo
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Mouse Enh.'), unitless	0.257	0.12	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Promoters'), unitless	0.538	0.436	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Regulatory'), unitless	0.183	0.125	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Species Clf.'), unitless	0.391	0.207	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Splice Sites'), unitless	0.298	0.0657	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'TF Binding'), unitless	0.254	0.0386	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Virus/Phage'), unitless	0.267	0.0941	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'lncRNA'), unitless	0.178	0.0189	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs), unitless	0.324	0.194	LucaOne
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; micro average: mean over all 100 tasks), unitless	0.313	0.232	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Chromatin Acc.'), unitless	0.15	0.14	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Coding/NC'), unitless	0.375	0.293	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'DNA Methyl.'), unitless	0.0148	0.0208	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Enhancers'), unitless	0.16	0.165	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Histone Mod.'), unitless	0.0957	0.108	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Mouse Enh.'), unitless	0.0698	0.0568	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Promoters'), unitless	0.287	0.279	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Regulatory'), unitless	0.0352	0.0788	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Species Clf.'), unitless	0.176	0.103	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Splice Sites'), unitless	0.0774	0.0643	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'TF Binding'), unitless	0.092	0.0084	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Virus/Phage'), unitless	0.0798	-0.00343	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'lncRNA'), unitless	0.0554	0.00964	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs), unitless	0.128	0.102	LucaOne
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; micro average: mean over all 100 tasks), unitless	0.136	0.126	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Chromatin Acc.'), unitless	0.662	0.284	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Coding/NC'), unitless	0.901	0.719	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'DNA Methyl.'), unitless	0.323	0.0732	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Enhancers'), unitless	0.479	0.37	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Histone Mod.'), unitless	0.533	0.397	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Mouse Enh.'), unitless	0.597	0.287	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Promoters'), unitless	0.746	0.537	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Regulatory'), unitless	0.492	0.269	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Species Clf.'), unitless	0.662	0.285	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Splice Sites'), unitless	0.636	0.16	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'TF Binding'), unitless	0.605	0.173	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Virus/Phage'), unitless	0.519	0.205	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'lncRNA'), unitless	0.508	0.119	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; macro average: unweighted mean of the 13 category MCCs), unitless	0.589	0.298	LucaOne
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; micro average: mean over all 100 tasks), unitless	0.579	0.342	LucaOne

What disagrees

Other studies: rank LucaOne above Evo in 40 of 45 comparisons
This study: ranks Evo above LucaOne in 6 of 6 comparisons

Compared: Evo, LucaOne
Study that disagrees: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning, Zero-shot benchmarking of RNA language models: Supplementary Information
Other studies: GENEB leaderboard (leaderboard.json, 13 task-type aggregates)
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (12)
Study	Tool	Protocol or benchmark	Metric	Value
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.588±0.056
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.586
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.132±0.088
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.13
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.179±0.114
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.178
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.559±0.055
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.088±0.082
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.066
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.118±0.105
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (D17.6M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.091
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	LucaOne (G36.8M) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.544

In the other studies (90 results)

In the other studies
Study	Tool	Protocol or benchmark	Metric	Value
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Chromatin Acc.')	0.289
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Coding/NC')	0.539
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'DNA Methyl.')	0.0458
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Enhancers')	0.288
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Histone Mod.')	0.258
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'lncRNA')	0.0189
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs)	0.194
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; micro average: mean over all 100 tasks)	0.232
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Mouse Enh.')	0.12
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Promoters')	0.436
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Regulatory')	0.125
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Species Clf.')	0.207
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Splice Sites')	0.0657
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'TF Binding')	0.0386
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Virus/Phage')	0.0941
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Chromatin Acc.')	0.14
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Coding/NC')	0.293
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'DNA Methyl.')	0.0208
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Enhancers')	0.165
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Histone Mod.')	0.108
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'lncRNA')	0.00964
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs)	0.102
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; micro average: mean over all 100 tasks)	0.126
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Mouse Enh.')	0.0568
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Promoters')	0.279
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Regulatory')	0.0788
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Species Clf.')	0.103
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Splice Sites')	0.0643
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'TF Binding')	0.0084
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Virus/Phage')	-0.00343
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Chromatin Acc.')	0.284
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Coding/NC')	0.719
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'DNA Methyl.')	0.0732
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Enhancers')	0.37
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Histone Mod.')	0.397
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'lncRNA')	0.119
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; macro average: unweighted mean of the 13 category MCCs)	0.298
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; micro average: mean over all 100 tasks)	0.342
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Mouse Enh.')	0.287
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Promoters')	0.537
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Regulatory')	0.269
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Species Clf.')	0.285
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Splice Sites')	0.16
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'TF Binding')	0.173
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Virus/Phage')	0.205
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Chromatin Acc.')	0.478
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Coding/NC')	0.746
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'DNA Methyl.')	0.0665
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Enhancers')	0.304
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Histone Mod.')	0.25
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'lncRNA')	0.178
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs)	0.324
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; micro average: mean over all 100 tasks)	0.313
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Mouse Enh.')	0.257
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Promoters')	0.538
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Regulatory')	0.183
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Species Clf.')	0.391
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Splice Sites')	0.298
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'TF Binding')	0.254
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Virus/Phage')	0.267
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Chromatin Acc.')	0.15
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Coding/NC')	0.375
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'DNA Methyl.')	0.0148
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Enhancers')	0.16
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Histone Mod.')	0.0957
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'lncRNA')	0.0554
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs)	0.128
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; micro average: mean over all 100 tasks)	0.136
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Mouse Enh.')	0.0698
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Promoters')	0.287
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Regulatory')	0.0352
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Species Clf.')	0.176
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Splice Sites')	0.0774
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'TF Binding')	0.092
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Virus/Phage')	0.0798
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Chromatin Acc.')	0.662
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Coding/NC')	0.901
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'DNA Methyl.')	0.323
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Enhancers')	0.479
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Histone Mod.')	0.533
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'lncRNA')	0.508
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; macro average: unweighted mean of the 13 category MCCs)	0.589
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; micro average: mean over all 100 tasks)	0.579
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Mouse Enh.')	0.597
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Promoters')	0.746
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Regulatory')	0.492
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Species Clf.')	0.662
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Splice Sites')	0.636
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'TF Binding')	0.605
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	LucaOne-default-step36M (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Virus/Phage')	0.519

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-3156024cb53500b8 · All contested results · How findings are made and followed up