Configuration: Nucleotide Transformer 2.5B-parameter models, IA3 fine-tuning (Dalla-Torre et al. 2025) Protocol: Nucleotide Transformer IA3 fine-tuning run time (Dalla-Torre et al. 2025 Methods 'Hardware') Dataset: NT benchmark, all 18 tasks 50 Runtime (Mean wall time per IA3 fine-tuning run)minute · lower
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Fine-tuning time conflicts within the article. Methods 'Hardware' say all fine-tuning runs used one node of eight A100 GPUs and averaged 20 min (500M-parameter models) and 50 min (2.5B-parameter models); the Discussion says the IA3 technique allows even the largest models to be fine-tuned in under 15 min on a single GPU. Both are recorded; neither is preferred.Nucleotide Transformer: building and evaluating robust foundation models for human genomics (Nature Methods) · Methods, 'Data preparation > Hardware', paragraph 1; Discussion, paragraph 3
Methods, coverage and source NT 2.5B-parameter models IA3 fine-tuning run time (Dalla-Torre et al. 2025)
gfm-guide-20261014-protocol-nt-ia3-finetune-runtime
Aggregation: Not reported
Nucleotide Transformer: building and evaluating robust foundation models for human genomics (Nature Methods) · Methods, 'Data preparation > Hardware', paragraph 1: 'On average, a fine-tuning run lasted 20 min for the 500M-parameter models and 50 min for the 2.5B-parameter models.' Configuration: Nucleotide Transformer 500M-parameter models, IA3 fine-tuning (Dalla-Torre et al. 2025) Protocol: Nucleotide Transformer IA3 fine-tuning run time (Dalla-Torre et al. 2025 Methods 'Hardware') Dataset: NT benchmark, all 18 tasks 20 Runtime (Mean wall time per IA3 fine-tuning run)minute · lower
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Fine-tuning time conflicts within the article. Methods 'Hardware' say all fine-tuning runs used one node of eight A100 GPUs and averaged 20 min (500M-parameter models) and 50 min (2.5B-parameter models); the Discussion says the IA3 technique allows even the largest models to be fine-tuned in under 15 min on a single GPU. Both are recorded; neither is preferred.Nucleotide Transformer: building and evaluating robust foundation models for human genomics (Nature Methods) · Methods, 'Data preparation > Hardware', paragraph 1; Discussion, paragraph 3
Methods, coverage and source NT 500M-parameter models IA3 fine-tuning run time (Dalla-Torre et al. 2025)
gfm-guide-20261014-protocol-nt-ia3-finetune-runtime
Aggregation: Not reported
Nucleotide Transformer: building and evaluating robust foundation models for human genomics (Nature Methods) · Methods, 'Data preparation > Hardware', paragraph 1: 'On average, a fine-tuning run lasted 20 min for the 500M-parameter models and 50 min for the 2.5B-parameter models.' Configuration: BPNet (large), trained from scratch (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.683 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source BPNet (large) (control) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'BPNet (large)', approach 'control', column 'Average performance' Configuration: BPNet (original), trained from scratch (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.665 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source BPNet (original) (control) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'BPNet (original)', approach 'control', column 'Average performance' Configuration: BPNet (X-large), trained from scratch (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.566 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source BPNet (X-large) (control) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'BPNet (X-large)', approach 'control', column 'Average performance' Configuration: DNABERT-1, fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.511 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source DNABERT-1 (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'DNABERT-1', approach 'Finetuning', column 'Average performance' Configuration: DNABERT-2, parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.723 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source DNABERT-2 (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'DNABERT-2', approach 'Finetuning', column 'Average performance' Configuration: Enformer, parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.684 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source Enformer (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'Enformer', approach 'Finetuning', column 'Average performance' Configuration: HyenaDNA-1KB, parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.668 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source HyenaDNA-1KB (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'HyenaDNA-1KB', approach 'Finetuning', column 'Average performance' Configuration: HyenaDNA-32KB, parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.691 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Independent external evaluation · Source checked Methods, coverage and source HyenaDNA-32KB (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'HyenaDNA-32KB', approach 'Finetuning', column 'Average performance' Configuration: NT-1000G (2.5B), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.738 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-1000G (2.5B) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-1000G (2.5B)', approach 'Finetuning', column 'Average performance' Configuration: NT-1000G (2.5B), probing (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.613 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-1000G (2.5B) (Probing) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-1000G (2.5B)', approach 'Probing', column 'Average performance' Configuration: NT-1000G (500M), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.729 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-1000G (500M) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-1000G (500M)', approach 'Finetuning', column 'Average performance' Configuration: NT-1000G (500M), probing (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.583 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-1000G (500M) (Probing) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-1000G (500M)', approach 'Probing', column 'Average performance' Configuration: NT 2.5B Randomly Initialized, trained from scratch (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.399 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT 2.5B Randomly Initialized (Training from scratch) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT 2.5B Randomly Initialized', approach 'Training from scratch', column 'Average performance' Configuration: NT 500M Randomly Initialized, trained from scratch (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.355 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT 500M Randomly Initialized (Training from scratch) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT 500M Randomly Initialized', approach 'Training from scratch', column 'Average performance' Configuration: NT-HumanRef (500M), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.733 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-HumanRef (500M) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-HumanRef (500M)', approach 'Finetuning', column 'Average performance' Configuration: NT-HumanRef (500M), probing (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.534 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-HumanRef (500M) (Probing) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-HumanRef (500M)', approach 'Probing', column 'Average performance' Configuration: NT-Multispecies (2.5B), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.755 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-Multispecies (2.5B) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-Multispecies (2.5B)', approach 'Finetuning', column 'Average performance' Configuration: NT-Multispecies (2.5B), probing (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.644 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-Multispecies (2.5B) (Probing) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-Multispecies (2.5B)', approach 'Probing', column 'Average performance' Configuration: NT-Multispecies-v2 (100M), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.748 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-Multispecies-v2 (100M) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-Multispecies-v2 (100M)', approach 'Finetuning', column 'Average performance' Configuration: NT-Multispecies-v2 (250M), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.769 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-Multispecies-v2 (250M) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-Multispecies-v2 (250M)', approach 'Finetuning', column 'Average performance' Configuration: NT-Multispecies-v2 (500M), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.762 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-Multispecies-v2 (500M) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-Multispecies-v2 (500M)', approach 'Finetuning', column 'Average performance' Configuration: NT-Multispecies-v2 (50M), parameter-efficient fine-tuning (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.732 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source NT-Multispecies-v2 (50M) (Finetuning) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'NT-Multispecies-v2 (50M)', approach 'Finetuning', column 'Average performance' Configuration: Raw Probe, probing of raw sequence (NT benchmark) Protocol: NT benchmark normalised summed performance (Supplementary Table 9) Dataset: NT benchmark, all 18 tasks 0.391 MCC (mean over three task categories of median cross-validation MCC)unitless · higher
Uncertainty: Not reported by the source
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Methods, coverage and source Raw Probe (Probing) NT benchmark normalised summed performance
lit-genomics-20261011-ntbench-protocol-normalised-summed-performance
Aggregation: Not reported
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12) · Supplementary Table 9, row 'Raw Probe', approach 'Probing', column 'Average performance'