Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) ranks Nucleotide Transformer above Nucleotide Transformer v2 in 37 of 44 comparisons; 2 of 2 other studies rank Nucleotide Transformer v2 above Nucleotide Transformer.
2 papers rank Nucleotide Transformer v2 above Nucleotide Transformer in 33 of their 46 comparisons and Nucleotide Transformer higher in 7; 6 are ties. GENEB leaderboard ranks Nucleotide Transformer above Nucleotide Transformer v2 in 37 of its 45 comparisons and Nucleotide Transformer v2 higher in 7; 1 is a tie.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks Nucleotide Transformer above RNAErnie in 7 of 7 comparisons; RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Nucleotide Transformer above RNAErnie in 7 of their 19 comparisons and RNAErnie higher in 4; 8 are ties. Of those papers, RNAGym fitness leaderboard ranks RNAErnie higher on its own (4 of 4).