Result

Macro F1 (ViroBench DNA viruses, taxonomy, G-Split) of GENERator-v2-prokaryote-1.2b-Base (ViroBench classification) on ViroBench DNA viruses, G-Split test set

2.30 (0.66)% Macro F1 (ViroBench DNA viruses, taxonomy, G-Split)

Evidence concern: excluded from comparisons

Table 25 prints one Macro-F1 per model for the taxonomy columns. For 143 of the 144 model-by-split taxonomy cells on the ALL-virus set, the printed value equals the mean of the five per-rank values in Table 26 rounded to two decimals. The exception is RNA-FM under the genus-disjoint split: Table 25 prints 48.67 and the mean of the Table 26 ranks (59.96, 65.35, 37.58, 58.78, 43.82) is 53.10. One of the two tables is wrong for that row, so RNA-FM ALL taxonomy G-Split should not be used in a comparison without checking the published code.ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks · Table 25, row 'RNA-FM', column 'ALL viruses / Taxonomy / G-Split'; Table 26, row 'RNA-FM'

Table 3 lists NT v3 under phage-specific pretraining coverage (P) and Table 9 lists it as non-viral (N). Table 25 groups the NTv3 rows under phage-specific coverage, which is what the configurations record. Pretraining coverage is the paper's main explanatory variable, so NT v3 rows should not be used to support a coverage-versus-performance claim.ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks · Table 3 row 'NT v3'; Table 9 row 'NT v3'; Table 25 section 'DNA Foundation Models (Phage-specific Coverage)'

Methods

Tested configuration: GENERator-v2-prokaryote-1.2b-Base (ViroBench classification)
Protocol: ViroBench DNA viruses taxonomy classification, G-Split, Macro-F1
Dataset: ViroBench DNA viruses, G-Split test set
Procedure: ViroBench DNA viruses taxonomy classification, G-Split, Macro-F1
Evaluation: GENERator-v2-prokaryote-1.2b-Base on ViroBench DNA viruses taxonomy classification (G-Split)
Coverage: Not reported scored / Not reported eligible
Uncertainty: SD 0.66
Evidence: Independent external evaluation · source checkedViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks · Table 25, row 'GENERator-v2-prokaryote-1.2b-Base', column 'DNA viruses / Taxonomy / G-Split' (text-layer line 2603)

A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.

Reproduction

Split: G-Split test partition
Adaptation: Frozen embeddings with a standardised MLP head
Scoring implementation: Macro-F1 x100 as printed

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
Reported result 2.30 (0.66) Individual claims	ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks Original source ↗ Table 25, row 'GENERator-v2-prokaryote-1.2b-Base', column 'DNA viruses / Taxonomy / G-Split' (text-layer line 2603) Version: arXiv 2605.25388v1 (submitted 2026-05-25); 42-page PDF; KDD '26 header on every page Retrieved: 2026-10-11T21:24:29Z	source checked ["source-hash-verification","pdf-text-parse","independent-cell-check","rendered-source-visual-check"] · 2026-10-11 independent paper Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Extraction: Parsed the pdftotext -layout text layer of the pinned arXiv PDF with extract/extract_virobench.py; each row has a label and twelve 'mean (sd)' cells asserted. Table 25, text-layer line 2603. Independent review 2026-10-11: Re-downloaded the arXiv PDF and matched its SHA-256 and the text-layer hash, including against the archived gzip. Parsed every cell of Table 25 (74 rows x 12) and Table 4 (34 rows) with an independent parser and matched all 912 results by source_locator: printed values, numeric values, standard deviations and text-layer line numbers all match, with no duplicate or missing cell. Every Table 4 row also present in Table 25 agrees cell for cell, and the two rows unique to Table 4 agree with Table 19. Read 79 cells by eye from pages rendered at 150 and 400 dpi, covering both Table 25 pages, all four baselines, all coverage groups, both tasks, both splits, all three virus sets and models from 1M to 40B parameters: 79 of 79 confirmed. Field: `attributes.printed_value` Source artifact SHA-256: `22ad7d742eeba37dd7f7f407a483776ba15fc38dedfb8c27ec05f5db231f9e42` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact Extraction artifact SHA-256: `22ad7d742eeba37dd7f7f407a483776ba15fc38dedfb8c27ec05f5db231f9e42` Extraction artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

1 source record and release history

ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks · Original source · arXiv 2605.25388v1 (submitted 2026-05-25); 42-page PDF; KDD '26 header on every page

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: lit-microbial-20261011-virobench-result-generator-v2-prokaryote-1-2b-base-dna-taxonomy-g-split

metric: macro-f1
metric direction: higher
unit: percent
printed value: 2.30 (0.66)
source locator: Table 25, row 'GENERator-v2-prokaryote-1.2b-Base', column 'DNA viruses / Taxonomy / G-Split' (text-layer line 2603)
review: method: source-hash-verification; pdf-text-parse; independent-cell-check; rendered-source-visual-check; method note: Re-downloaded the arXiv PDF and matched its SHA-256 and the text-layer hash, including against the archived gzip. Parsed every cell of Table 25 (74 rows x 12) and Table 4 (34 rows) with an independent parser and matched all 912 results by source_locator: printed values, numeric values, standard deviations and text-layer line numbers all match, with no duplicate or missing cell. Every Table 4 row also present in Table 25 agrees cell for cell, and the two rows unique to Table 4 agree with Table 19. Read 79 cells by eye from pages rendered at 150 and 400 dpi, covering both Table 25 pages, all four baselines, all coverage groups, both tasks, both splits, all three virus sets and models from 1M to 40B parameters: 79 of 79 confirmed.; reviewer: claude; reviewer note: Claude review agent reviewer-mi-1, independent of the extractor lit-microbial-interactions; no human review claimed; date: 2026-10-11; artifact sha256: 22ad7d742eeba37dd7f7f407a483776ba15fc38dedfb8c27ec05f5db231f9e42; retrieval url: https://arxiv.org/pdf/2605.25388v1; note: Extraction: Parsed the pdftotext -layout text layer of the pinned arXiv PDF with extract/extract_virobench.py; each row has a label and twelve 'mean (sd)' cells asserted. Table 25, text-layer line 2603. Independent review 2026-10-11: Re-downloaded the arXiv PDF and matched its SHA-256 and the text-layer hash, including against the archived gzip. Parsed every cell of Table 25 (74 rows x 12) and Table 4 (34 rows) with an independent parser and matched all 912 results by source_locator: printed values, numeric values, standard deviations and text-layer line numbers all match, with no duplicate or missing cell. Every Table 4 row also present in Table 25 agrees cell for cell, and the two rows unique to Table 4 agree with Table 19. Read 79 cells by eye from pages rendered at 150 and 400 dpi, covering both Table 25 pages, all four baselines, all coverage groups, both tasks, both splits, all three virus sets and models from 1M to 40B parameters: 79 of 79 confirmed.
numeric value: 2.30
metric qualifier: ViroBench DNA viruses, taxonomy, G-Split
unit detail: Macro-F1 x100 as printed
uncertainty: type: standard_deviation; value: 0.66; printed: (0.66); scope: Across hyperparameter settings (window configurations and learning rates), Section 4.1.2
printed value basis: Mean followed by the standard deviation in parentheses, as printed

Related records

evaluation: GENERator-v2-prokaryote-1.2b-Base on ViroBench DNA viruses taxonomy classification (G-Split)

Suggest a correction