Contested result

Nucleotide Transformer v2 and Evo: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above Nucleotide Transformer v2 in 6 of 6 comparisons; other studies rank Nucleotide Transformer v2 above Evo in 42 of 45.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

GENEB leaderboard ranks Nucleotide Transformer v2 above Evo in 42 of its 45 comparisons and Evo higher in 3. Zero-shot benchmarking of RNA language… ranks Evo above Nucleotide Transformer v2 in all 6 of its comparisons.

Rank Nucleotide Transformer v2 higher (1): GENEB leaderboard
Rank Evo higher (1): Zero-shot benchmarking of RNA language… (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Nucleotide Transformer v2 higherTieEvo higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks Nucleotide Transformer v2 higher	Ranks Evo higher	Ties
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning and 1 more record · DOI	Disagrees	0	6	0
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) · Repository	Agrees	42	3	0

Comparisons in Zero-shot benchmarking of RNA language… (6)

Protocol	Dataset	Metric	Nucleotide Transformer v2	Evo	Higher
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; mean over assays), unitless	0.557	0.588	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	AUROC (PLL-D; median over assays), unitless	0.547	0.586	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; mean over assays), unitless	0.082	0.132	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	MCC (PLL-D; median over assays), unitless	0.075	0.13	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; mean over assays), unitless	0.111	0.179	Evo
Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	RNAGym ncRNA DMS assays (31 assays) (Wang et al. 2026)	Absolute spearman correlation (PLL-D; median over assays), unitless	0.085	0.178	Evo

Comparisons in GENEB leaderboard (45)

Protocol	Dataset	Metric	Nucleotide Transformer v2	Evo	Higher
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Chromatin Acc.'), unitless	0.481	0.289	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Coding/NC'), unitless	0.587	0.539	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'DNA Methyl.'), unitless	0.0804	0.0458	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Enhancers'), unitless	0.3	0.288	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Histone Mod.'), unitless	0.261	0.258	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Mouse Enh.'), unitless	0.171	0.12	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Promoters'), unitless	0.545	0.436	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Regulatory'), unitless	0.182	0.125	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Species Clf.'), unitless	0.564	0.207	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Splice Sites'), unitless	0.129	0.0657	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'TF Binding'), unitless	0.242	0.0386	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'Virus/Phage'), unitless	0.147	0.0941	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; category 'lncRNA'), unitless	0.0528	0.0189	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs), unitless	0.286	0.194	Nucleotide Transformer v2
GENEB leaderboard, 10-shot, MCC aggregates	GENEB 100 tasks	MCC (10-shot; micro average: mean over all 100 tasks), unitless	0.293	0.232	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Chromatin Acc.'), unitless	0.186	0.14	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Coding/NC'), unitless	0.284	0.293	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'DNA Methyl.'), unitless	0.0223	0.0208	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Enhancers'), unitless	0.159	0.165	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Histone Mod.'), unitless	0.115	0.108	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Mouse Enh.'), unitless	0.0424	0.0568	Evo
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Promoters'), unitless	0.36	0.279	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Regulatory'), unitless	0.113	0.0788	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Species Clf.'), unitless	0.304	0.103	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Splice Sites'), unitless	0.0704	0.0643	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'TF Binding'), unitless	0.0992	0.0084	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'Virus/Phage'), unitless	0.0356	-0.00343	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; category 'lncRNA'), unitless	0.037	0.00964	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs), unitless	0.136	0.102	Nucleotide Transformer v2
GENEB leaderboard, 1-shot, MCC aggregates	GENEB 100 tasks	MCC (1-shot; micro average: mean over all 100 tasks), unitless	0.158	0.126	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Chromatin Acc.'), unitless	0.652	0.284	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Coding/NC'), unitless	0.812	0.719	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'DNA Methyl.'), unitless	0.339	0.0732	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Enhancers'), unitless	0.469	0.37	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Histone Mod.'), unitless	0.473	0.397	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Mouse Enh.'), unitless	0.492	0.287	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Promoters'), unitless	0.722	0.537	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Regulatory'), unitless	0.457	0.269	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Species Clf.'), unitless	0.747	0.285	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Splice Sites'), unitless	0.47	0.16	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'TF Binding'), unitless	0.55	0.173	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'Virus/Phage'), unitless	0.482	0.205	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; category 'lncRNA'), unitless	0.337	0.119	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; macro average: unweighted mean of the 13 category MCCs), unitless	0.537	0.298	Nucleotide Transformer v2
GENEB leaderboard, full training data, MCC aggregates	GENEB 100 tasks	MCC (full training data; micro average: mean over all 100 tasks), unitless	0.524	0.342	Nucleotide Transformer v2

What disagrees

Other studies: rank Nucleotide Transformer v2 above Evo in 42 of 45 comparisons
This study: ranks Evo above Nucleotide Transformer v2 in 6 of 6 comparisons

Compared: Nucleotide Transformer v2, Evo
Study that disagrees: Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning, Zero-shot benchmarking of RNA language models: Supplementary Information
Other studies: GENEB leaderboard (leaderboard.json, 13 task-type aggregates)
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (12)
Study	Tool	Protocol or benchmark	Metric	Value
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.588±0.056
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.586
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.132±0.088
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.13
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.179±0.114
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	EVO1.5 (8k_base) (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.178
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	NT (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (mean over assays)	AUROC (PLL-D; mean over assays)	0.557±0.056
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	NT (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, AUC (median over assays)	AUROC (PLL-D; median over assays)	0.547
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	NT (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (mean over assays)	MCC (PLL-D; mean over assays)	0.082±0.076
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	NT (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, MCC (median over assays)	MCC (PLL-D; median over assays)	0.075
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	NT (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (mean over assays)	Absolute spearman correlation (PLL-D; mean over assays)	0.111±0.109
Zero-shot benchmarking of RNA language models: Supplementary Information; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning	NT (zero-shot, Wang et al. 2026)	Wang et al. 2026 zero-shot fitness, RNAGym ncRNA DMS assays (31 assays), PLL-D, SR (median over assays)	Absolute spearman correlation (PLL-D; median over assays)	0.085

In the other studies (90 results)

In the other studies
Study	Tool	Protocol or benchmark	Metric	Value
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Chromatin Acc.')	0.289
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Coding/NC')	0.539
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'DNA Methyl.')	0.0458
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Enhancers')	0.288
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Histone Mod.')	0.258
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'lncRNA')	0.0189
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs)	0.194
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; micro average: mean over all 100 tasks)	0.232
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Mouse Enh.')	0.12
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Promoters')	0.436
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Regulatory')	0.125
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Species Clf.')	0.207
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Splice Sites')	0.0657
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'TF Binding')	0.0386
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Virus/Phage')	0.0941
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Chromatin Acc.')	0.14
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Coding/NC')	0.293
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'DNA Methyl.')	0.0208
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Enhancers')	0.165
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Histone Mod.')	0.108
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'lncRNA')	0.00964
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs)	0.102
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; micro average: mean over all 100 tasks)	0.126
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Mouse Enh.')	0.0568
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Promoters')	0.279
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Regulatory')	0.0788
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Species Clf.')	0.103
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Splice Sites')	0.0643
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'TF Binding')	0.0084
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Virus/Phage')	-0.00343
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Chromatin Acc.')	0.284
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Coding/NC')	0.719
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'DNA Methyl.')	0.0732
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Enhancers')	0.37
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Histone Mod.')	0.397
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'lncRNA')	0.119
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; macro average: unweighted mean of the 13 category MCCs)	0.298
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; micro average: mean over all 100 tasks)	0.342
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Mouse Enh.')	0.287
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Promoters')	0.537
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Regulatory')	0.269
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Species Clf.')	0.285
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Splice Sites')	0.16
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'TF Binding')	0.173
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	evo-1-131k (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Virus/Phage')	0.205
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-100m-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'DNA Methyl.')	0.0223
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-100m-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Regulatory')	0.113
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Histone Mod.')	0.261
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'lncRNA')	0.0528
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Coding/NC')	0.284
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Enhancers')	0.159
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Chromatin Acc.')	0.652
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Coding/NC')	0.812
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'DNA Methyl.')	0.339
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Enhancers')	0.469
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'lncRNA')	0.337
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; macro average: unweighted mean of the 13 category MCCs)	0.537
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; micro average: mean over all 100 tasks)	0.524
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Promoters')	0.722
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Regulatory')	0.457
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Species Clf.')	0.747
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Splice Sites')	0.47
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-250m-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Virus/Phage')	0.482
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Chromatin Acc.')	0.481
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Coding/NC')	0.587
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'DNA Methyl.')	0.0804
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Enhancers')	0.3
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; macro average: unweighted mean of the 13 category MCCs)	0.286
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; micro average: mean over all 100 tasks)	0.293
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Mouse Enh.')	0.171
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Promoters')	0.545
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Species Clf.')	0.564
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Splice Sites')	0.129
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'TF Binding')	0.242
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Chromatin Acc.')	0.186
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Histone Mod.')	0.115
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'lncRNA')	0.037
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; macro average: unweighted mean of the 13 category MCCs)	0.136
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; micro average: mean over all 100 tasks)	0.158
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Mouse Enh.')	0.0424
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Promoters')	0.36
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Species Clf.')	0.304
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Splice Sites')	0.0704
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'TF Binding')	0.0992
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Histone Mod.')	0.473
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'Mouse Enh.')	0.492
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-3mer-MS (GENEB leaderboard)	GENEB leaderboard, full training data, MCC aggregates	MCC (full training data; category 'TF Binding')	0.55
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Regulatory')	0.182
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-MS (GENEB leaderboard)	GENEB leaderboard, 10-shot, MCC aggregates	MCC (10-shot; category 'Virus/Phage')	0.147
GENEB leaderboard (leaderboard.json, 13 task-type aggregates)	NT-v2-50m-MS (GENEB leaderboard)	GENEB leaderboard, 1-shot, MCC aggregates	MCC (1-shot; category 'Virus/Phage')	0.0356

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-a1e76daf59cd0666 · All contested results · How findings are made and followed up