rewirebio.iobenchmarks
Source

Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages) · Peer-reviewed

DOI: 10.1038/s41592-024-02523-z

What this source contributed

Records in this release that cite this source. A record can cite several sources.

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

ValueMetricModel or configurationDatasetEvaluation
0.683 unitlessMCC (mean over three task categories of median cross-validation MCC)BPNet (large), trained from scratch (NT benchmark)NT benchmark, all 18 tasksBPNet (large) (control) NT benchmark normalised summed performance
0.492 (± 0.008) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark EnhancerBPNet (large) on NT benchmark Enhancer (probing)
0.454 (± 0.008) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Enhancer typesBPNet (large) on NT benchmark Enhancer types (probing)
0.487 (± 0.014) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H2AFZBPNet (large) on NT benchmark H2AFZ (probing)
0.459 (± 0.012) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K4me1BPNet (large) on NT benchmark H3K4me1 (probing)
0.427 (± 0.025) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K4me2BPNet (large) on NT benchmark H3K4me2 (probing)
0.445 (± 0.049) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K4me3BPNet (large) on NT benchmark H3K4me3 (probing)
0.298 (± 0.033) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K9acBPNet (large) on NT benchmark H3K9ac (probing)
0.234 (± 0.037) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K9me3BPNet (large) on NT benchmark H3K9me3 (probing)
0.214 (± 0.037) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K27acBPNet (large) on NT benchmark H3K27ac (probing)
0.551 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K27me3BPNet (large) on NT benchmark H3K27me3 (probing)
0.57 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H3K36me3BPNet (large) on NT benchmark H3K36me3 (probing)
0.525 (± 0.038) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark H4K20me1BPNet (large) on NT benchmark H4K20me1 (probing)
0.672 (± 0.023) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Promoter allBPNet (large) on NT benchmark Promoter all (probing)
0.672 (± 0.043) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Promoter non-TATABPNet (large) on NT benchmark Promoter non-TATA (probing)
0.826 (± 0.017) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Promoter TATABPNet (large) on NT benchmark Promoter TATA (probing)
0.925 (± 0.031) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Splice acceptorBPNet (large) on NT benchmark Splice acceptor (probing)
0.93 (± 0.021) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Splice donorBPNet (large) on NT benchmark Splice donor (probing)
0.865 (± 0.026) unitlessMCC (median over 10 cross-validation folds)BPNet (large), trained from scratch (NT benchmark)NT benchmark Splice site allBPNet (large) on NT benchmark Splice site all (probing)
0.665 unitlessMCC (mean over three task categories of median cross-validation MCC)BPNet (original), trained from scratch (NT benchmark)NT benchmark, all 18 tasksBPNet (original) (control) NT benchmark normalised summed performance
0.488 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark EnhancerBPNet (original) on NT benchmark Enhancer (probing)
0.449 (± 0.006) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Enhancer typesBPNet (original) on NT benchmark Enhancer types (probing)
0.473 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H2AFZBPNet (original) on NT benchmark H2AFZ (probing)
0.436 (± 0.008) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K4me1BPNet (original) on NT benchmark H3K4me1 (probing)
0.427 (± 0.036) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K4me2BPNet (original) on NT benchmark H3K4me2 (probing)
0.445 (± 0.047) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K4me3BPNet (original) on NT benchmark H3K4me3 (probing)
0.336 (± 0.034) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K9acBPNet (original) on NT benchmark H3K9ac (probing)
0.298 (± 0.03) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K9me3BPNet (original) on NT benchmark H3K9me3 (probing)
0.296 (± 0.046) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K27acBPNet (original) on NT benchmark H3K27ac (probing)
0.543 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K27me3BPNet (original) on NT benchmark H3K27me3 (probing)
0.548 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H3K36me3BPNet (original) on NT benchmark H3K36me3 (probing)
0.531 (± 0.025) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark H4K20me1BPNet (original) on NT benchmark H4K20me1 (probing)
0.696 (± 0.026) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Promoter allBPNet (original) on NT benchmark Promoter all (probing)
0.717 (± 0.023) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Promoter non-TATABPNet (original) on NT benchmark Promoter non-TATA (probing)
0.848 (± 0.042) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Promoter TATABPNet (original) on NT benchmark Promoter TATA (probing)
0.859 (± 0.038) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Splice acceptorBPNet (original) on NT benchmark Splice acceptor (probing)
0.92 (± 0.014) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Splice donorBPNet (original) on NT benchmark Splice donor (probing)
0.793 (± 0.072) unitlessMCC (median over 10 cross-validation folds)BPNet (original), trained from scratch (NT benchmark)NT benchmark Splice site allBPNet (original) on NT benchmark Splice site all (probing)
0.566 unitlessMCC (mean over three task categories of median cross-validation MCC)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark, all 18 tasksBPNet (X-large) (control) NT benchmark normalised summed performance
0.501 (± 0.013) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark EnhancerBPNet (X-large) on NT benchmark Enhancer (probing)
0.455 (± 0.012) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark Enhancer typesBPNet (X-large) on NT benchmark Enhancer types (probing)
0.475 (± 0.011) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H2AFZBPNet (X-large) on NT benchmark H2AFZ (probing)
0.457 (± 0.013) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K4me1BPNet (X-large) on NT benchmark H3K4me1 (probing)
0.341 (± 0.038) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K4me2BPNet (X-large) on NT benchmark H3K4me2 (probing)
0.338 (± 0.076) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K4me3BPNet (X-large) on NT benchmark H3K4me3 (probing)
0.235 (± 0.06) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K9acBPNet (X-large) on NT benchmark H3K9ac (probing)
0.216 (± 0.085) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K9me3BPNet (X-large) on NT benchmark H3K9me3 (probing)
0.16 (± 0.042) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K27acBPNet (X-large) on NT benchmark H3K27ac (probing)
0.542 (± 0.009) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K27me3BPNet (X-large) on NT benchmark H3K27me3 (probing)
0.562 (± 0.011) unitlessMCC (median over 10 cross-validation folds)BPNet (X-large), trained from scratch (NT benchmark)NT benchmark H3K36me3BPNet (X-large) on NT benchmark H3K36me3 (probing)

Models, benchmarks and other records

Models (1)

Methods (1)

Configurations (23)

Benchmarks (1)

Protocols (37)

Datasets (19)

Evaluations (419)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

23 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statementOriginal source and locationReview and provenance
attributes.artifact_sha256
ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.artifact_sha256

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Artifact url
https://static-content.springer.com/esm/art%3A10.1038%2Fs41592-024-02523-z/MediaObjects/41592_2024_2523_MOESM1_ESM.pdf
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.artifact_url

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Doi
10.1038/s41592-024-02523-z
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.doi

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Licence
CC-BY-NC-ND-4.0
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.licence

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
3 values
  • Developer paper: the authors are InstaDeep and NVIDIA employees and developed the Nucleotide Transformer models (Competing interests). NT rows are author_reported.
  • Main-text Figure 2 and Extended Data are not read; the supplementary tables print the same per-task medians with standard deviations.
  • Supplementary Table 9 lists a DNABERT-1 fine-tuning average that has no per-task row in Supplementary Table 7.
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Media type
application/pdf
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.media_type

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Publication status
peer_reviewed
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.publication_status

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Retrieved at
2026-10-11T01:37:37Z
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.retrieved_at

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Review: date
2026-10-11
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.review.date

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Review: method note
Artifact re-retrieved or unpacked and SHA-256 matched.
Source metadata
Nucleotide Transformer: Supplementary Information (Supplementary Tables 1-12)

Original source ↗

No field-specific location recorded

Version: Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
Retrieved: 2026-10-11T01:37:37Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.review.method_note

Source artifact SHA-256: ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Primary source retrieved and hashed for the 2026-10-11 genomics literature pass.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)
Technical metadata and extraction receipts

Stable ID: lit-genomics-20261011-ntbench-source-dallatorre2025-si

areas
dna-genomes
contexts
research
url
https://doi.org/10.1038/s41592-024-02523-z
artifact url
https://static-content.springer.com/esm/art%3A10.1038%2Fs41592-024-02523-z/MediaObjects/41592_2024_2523_MOESM1_ESM.pdf
version
Nature Methods 22, 287-297 (published online 2024-11-28); Supplementary Information PDF (MOESM1, 25 pages)
retrieved at
2026-10-11T01:37:37Z
artifact sha256
ef0f357a1289aad1910059d455579e242a9a82ab3e32ec38d8ea2584effc436e
doi
10.1038/s41592-024-02523-z
publication status
peer_reviewed
licence
CC-BY-NC-ND-4.0
media type
application/pdf
limitations
Developer paper: the authors are InstaDeep and NVIDIA employees and developed the Nucleotide Transformer models (Competing interests). NT rows are author_reported.; Main-text Figure 2 and Extended Data are not read; the supplementary tables print the same per-task medians with standard deviations.; Supplementary Table 9 lists a DNABERT-1 fine-tuning average that has no per-task row in Supplementary Table 7.
review
method: source-hash-verification; method note: Artifact re-retrieved or unpacked and SHA-256 matched.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-2, independent of the extractor lit-genomics; no human review claimed; date: 2026-10-11
Related records

    Suggest a correction