rewirebio.iobenchmarks
Resolved finding

AlphaGenome and Borzoi: resolved

  • Low severity
  • Confirmed error

The author labelled upstream issue #2 a bug on 2026-10-20 and reported that rerunning with the 1-based position substantially improved accuracy, which confirms the error behind the discrepancy. The 42 AlphaGenome results of Alavi 2026 (nine evaluations: three modules on the multi-gene set, the OPA1 evaluation sets and the extended OPA1 positive calls) are now disputed and the article carries an evidence concern naming those evaluations, so the discrepancy rules no longer compare them. Their printed values are kept. Superseding results will be added from the revised preprint.

Resolved: confirmed error on 23 October 2026.

The compared results were withdrawn from the database (marked disputed) after the error was confirmed, so the check no longer finds this disagreement and the comparison between studies is not shown. The follow-up below records how it was resolved.

What disagreed

Compared
AlphaGenome, Borzoi
Study that disagrees
Other studies
Earlier IDs
discrepancy-ead251c24ba272c1

Follow-up

Current state: Confirmed error. Settled.

  1. 19 October 2026: Reported to the authors · marcel-2c8/Probing-genomic-foundation-models-2026#2

    State after this step: Awaiting a correction

    Both AlphaGenome scripts subtract 1 from a 1-based variant coordinate before calling AlphaGenome's 1-based Variant API, so every variant was scored one base from its real position. The author accepted this and reports that rerunning with the variant set to pos_1based substantially improved accuracy, so the stored sensitivity and specificity do not describe the intended variants.

    Authors' response: On 2026-10-20 the author labelled the issue a bug and replied that running the splice scorer with the variant set to pos_1based substantially improved accuracy, promising updated scripts in calendar week 44 and a revised manuscript with updated data in early November 2026.

    Waiting for: The author's corrected scripts and the revised preprint with rerun results, expected by 7 November 2026.

    Next check: 10 November 2026

  2. 23 October 2026: Corrected the records · marcel-2c8/Probing-genomic-foundation-models-2026#2

    State after this step: Confirmed error

    The author labelled upstream issue #2 a bug on 2026-10-20 and reported that rerunning with the 1-based position substantially improved accuracy, which confirms the error behind the discrepancy. The 42 AlphaGenome results of Alavi 2026 (nine evaluations: three modules on the multi-gene set, the OPA1 evaluation sets and the extended OPA1 positive calls) are now disputed and the article carries an evidence concern naming those evaluations, so the discrepancy rules no longer compare them. Their printed values are kept. Superseding results will be added from the revised preprint.

    Authors' response: On 2026-10-20 the author labelled the issue a bug and replied that running the splice scorer with the variant set to pos_1based substantially improved accuracy, promising updated scripts in calendar week 44 and a revised manuscript with updated data in early November 2026.

    Waiting for: The revised preprint with rerun AlphaGenome results, to add as superseding results, expected by 7 November 2026.

    Next check: 10 November 2026

    Resolved by: rna-splicing-20261011-alavi2026-result-multigene-alphagenome-either-module-accuracy, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-either-module-ppv, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-either-module-sensitivity, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-either-module-specificity, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-either-module-type-i-error, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-either-module-type-ii-error, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-junction-scorer-accuracy, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-junction-scorer-ppv, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-junction-scorer-sensitivity, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-junction-scorer-specificity, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-junction-scorer-type-i-error, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-junction-scorer-type-ii-error, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-splice-scorer-accuracy, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-splice-scorer-ppv, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-splice-scorer-sensitivity, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-splice-scorer-specificity, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-splice-scorer-type-i-error, rna-splicing-20261011-alavi2026-result-multigene-alphagenome-splice-scorer-type-ii-error, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-either-module-accuracy, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-either-module-ppv, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-either-module-sensitivity, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-either-module-specificity, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-either-module-type-i-error, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-either-module-type-ii-error, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-junction-scorer-accuracy, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-junction-scorer-ppv, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-junction-scorer-sensitivity, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-junction-scorer-specificity, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-junction-scorer-type-i-error, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-junction-scorer-type-ii-error, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-splice-scorer-accuracy, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-splice-scorer-ppv, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-splice-scorer-sensitivity, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-splice-scorer-specificity, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-splice-scorer-type-i-error, rna-splicing-20261011-alavi2026-result-opa1-evaluation-alphagenome-splice-scorer-type-ii-error, rna-splicing-20261011-alavi2026-result-opa1-extended-positive-calls-alphagenome-either-module-positive-mutations, rna-splicing-20261011-alavi2026-result-opa1-extended-positive-calls-alphagenome-either-module-positive-vus, rna-splicing-20261011-alavi2026-result-opa1-extended-positive-calls-alphagenome-junction-scorer-positive-mutations, rna-splicing-20261011-alavi2026-result-opa1-extended-positive-calls-alphagenome-junction-scorer-positive-vus, rna-splicing-20261011-alavi2026-result-opa1-extended-positive-calls-alphagenome-splice-scorer-positive-mutations, rna-splicing-20261011-alavi2026-result-opa1-extended-positive-calls-alphagenome-splice-scorer-positive-vus, Probing Genomic Foundation Models with Splice-Variant Perturbations (Alavi, bioRxiv 2026)

If you know more

If you have a correction to this record of how the finding was resolved, tell us through the contribution form.

Finding discrepancy-52a714eb6bb63442 · All contested results · How findings are made and followed up