rewirebio.iobenchmarks
Evidence claim

Bacformer Large embedding bug fixed on 2025-05-15

Known issue of Bacformer Large

A bug in the Bacformer Large code on Hugging Face, announced on 2025-05-15, significantly degraded the output embeddings; the README says it is fixed and that anyone who downloaded or cached the model before that date must re-download the latest revision before running evaluations

Any Bacformer Large number produced from a checkpoint cached before that date is not comparable with a number from the fixed code, and no source says which revision the preprint's runs used.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

8 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Claimed field
known_issue
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
1 values
  • The preprint does not state the checkpoint revision used for its runs, so whether its Bacformer Large numbers predate the fix cannot be settled from these sources.
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Any Bacformer Large number produced from a checkpoint cached before that date is not comparable with a number from the fixed code, and no source says which revision the preprint's runs used.
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Source location
Bacformer README 'News', entry dated 2025-05-15
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_locator

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed value
A bug in the Bacformer Large code on Hugging Face, announced on 2025-05-15, significantly degraded the output embeddings; the README says it is fixed and that anyone who downloaded or cached the model before that date must re-download the latest revision before running evaluations
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.value

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Description
Any Bacformer Large number produced from a checkpoint cached before that date is not comparable with a number from the fixed code, and no source says which revision the preprint's runs used.
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: description

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: subject
bacformer-20261115-model-bacformer-large
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: links:subject:bacformer-20261115-model-bacformer-large

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Name
Bacformer Large embedding bug fixed on 2025-05-15
Context-only references
Bacformer repository README at commit 61dca95

Original source ↗

Bacformer README 'News', entry dated 2025-05-15

Version: Commit 61dca9537429b95365d73d72a3e11c207df1ed6c
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: name

Source artifact SHA-256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: bacformer-20261115-claim-bacformer-large-bug-notice

areas
microbes-communities
contexts
research
field
known_issue
limitations
The preprint does not state the checkpoint revision used for its runs, so whether its Bacformer Large numbers predate the fix cannot be settled from these sources.
rationale
Any Bacformer Large number produced from a checkpoint cached before that date is not comparable with a number from the fixed code, and no source says which revision the preprint's runs used.
review
artifact sha256: 6e4fdabbd05c93ff63b0eb583ce6a9309eaa13998087cc32ddeaa6928025b235; method: transcription; source-hash-verification; ai-assisted-source-review; method note: Independent review pass. Each source artifact was retrieved again at review time and its SHA-256 compared with the source record; the quoted statement was then read at the recorded locator and compared with the recorded value. Access states were rechecked rather than taken from notes.md.; note: Statement re-read against the pinned artifact by a reviewer who did not extract this batch. Every artifact except the two preprint PDFs was downloaded again at review time and hashed to exactly the value its source record records, and is byte-identical to the copy kept in artifacts/.; retrieval url: https://raw.githubusercontent.com/macwiatrak/Bacformer/61dca9537429b95365d73d72a3e11c207df1ed6c/README.md; reviewed at: 2026-11-15T23:55:00Z; reviewer: claude; reviewer note: Claude (Opus 5) review agent, independent of the agent that extracted this batch; no human review claimed. Transcription was checked against the pinned sources. Nothing was reproduced and no value was recomputed.
source locator
Bacformer README 'News', entry dated 2025-05-15
value
A bug in the Bacformer Large code on Hugging Face, announced on 2025-05-15, significantly degraded the output embeddings; the README says it is fixed and that anyone who downloaded or cached the model before that date must re-download the latest revision before running evaluations
Related records

Suggest a correction