rewirebio.iobenchmarks
Result

Passing design clusters per accelerator hour (mean over targets of structurally distinct computational passers per GPU-hour, counting generation and common AlphaFold3 evaluation time, with 200 original attempts per target and failed attempts included; passing designs clustered within each model, protocol and target by Foldseek at TM 0.5; pass criterion: AF3 minimum interface PAE at most 1.5 A and DNA-target-aligned binder backbone RMSD at most 8 A, aligning on all target DNA atoms in both the fixed and unfixed modes) of BoltzGen, frozen project benchmark recipe on Frozen project benchmark DNA-binding target set, 3 targets

9.46 Passing design clusters per accelerator hour (mean over targets of structurally distinct computational passers per GPU-hour, counting generation and common AlphaFold3 evaluation time, with 200 original attempts per target and failed attempts included; passing designs clustered within each model, protocol and target by Foldseek at TM 0.5; pass criterion: AF3 minimum interface PAE at most 1.5 A and DNA-target-aligned binder backbone RMSD at most 8 A, aligning on all target DNA atoms in both the fixed and unfixed modes)

Methods

Tested configuration
BoltzGen, frozen project benchmark recipe
Protocol
Frozen project benchmark DNA binding, fixed DNA: distinct passers per GPU-hour
Dataset
Frozen project benchmark DNA-binding target set, 3 targets
Procedure
Frozen project benchmark DNA binding, fixed DNA: distinct passers per GPU-hour
Evaluation
BoltzGen (fixed DNA) on Frozen project benchmark DNA binding, fixed DNA: distinct passers per GPU-hour
Coverage
600/600 design attempts scored (3 targets times 200 original attempts)
Uncertainty
Not reported by the source: The table prints point values with no interval or spread.
Evidence
Author-reported evaluation · source checkedRFD4-Proteina Hugging Face model card at revision 621b785; RFD4-Proteina benchmarks/summary.csv at revision 621b785 · Hugging Face model card, 'Measured design efficiency' table, row 'DNA binders', column 'Best other method in this comparison' (printed as 'BoltzGen (fixed DNA) · 9.46')

A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.

Reproduction

Split
Evaluation only; every target scored
Adaptation
Native recipe as the benchmark methods supplement records it; no terminal selection by the common evaluation
Scoring implementation
mean_over_targets(3600 * passing_structure_clusters / (generation_gpu_s + af3_gpu_s)), published per 10 GPU-minutes in summary.csv and per GPU-hour on the model card

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

2 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Reported result
9.46
Individual claims
RFD4-Proteina benchmarks/summary.csv at revision 621b785

Original source ↗

Hugging Face model card, 'Measured design efficiency' table, row 'DNA binders', column 'Best other method in this comparison' (printed as 'BoltzGen (fixed DNA) · 9.46')

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: Hugging Face revision 621b785abf9d61dd24f4b81ae4dbd1330728f030
Retrieved: 2026-11-15T23:10:00Z

source checked

["independent-cell-check","deterministic-table-parse","source-hash-verification"] · 2026-11-15T00:00:00Z

author reported

Audit details

Every null and every missing_metadata entry on this result was checked against the sources and is genuinely unstated: no GPU model is named for the benchmark runs, no seed is published, and the passing-cluster count behind the rate is in benchmarks/source_data.csv, which this batch did not retrieve, so numerator is absent. No uncertainty is printed for any cell. The metric and unit concepts passing-design-clusters-per-accelerator-hour and count-per-hour were reviewed against data/vocab/metric.ttl and data/vocab/unit.ttl and both were accepted as genuinely missing; the metric was added to data/vocab/metric.ttl in this pass. This is a comparator cell. The printed method name was checked against the card and against the label column of the summary.csv row behind it, and the comparator configuration carries no model record because no primary source for the method was retrieved.

Field: attributes.printed_value

Source artifact SHA-256: 5c6d525d988d0df7fecb74396f86401c1ddb4c31fa83f00b357ba77d855d8a10

Hash scope: Hash scope not separately documented; inspect source record

Archive member: artifacts/hf_benchmarks_summary.csv.gz

Extraction artifact SHA-256: 38479d8e4c6e46ed7e1ec4ece0dd02c358ae46d73d56790c44be813201fd9b98

Extraction artifact

Reported result
9.46
Individual claims
RFD4-Proteina Hugging Face model card at revision 621b785

Original source ↗

Hugging Face model card, 'Measured design efficiency' table, row 'DNA binders', column 'Best other method in this comparison' (printed as 'BoltzGen (fixed DNA) · 9.46')

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: Hugging Face revision 621b785abf9d61dd24f4b81ae4dbd1330728f030
Retrieved: 2026-11-15T23:10:00Z

source checked

["independent-cell-check","deterministic-table-parse","source-hash-verification"] · 2026-11-15T00:00:00Z

author reported

Audit details

Every null and every missing_metadata entry on this result was checked against the sources and is genuinely unstated: no GPU model is named for the benchmark runs, no seed is published, and the passing-cluster count behind the rate is in benchmarks/source_data.csv, which this batch did not retrieve, so numerator is absent. No uncertainty is printed for any cell. The metric and unit concepts passing-design-clusters-per-accelerator-hour and count-per-hour were reviewed against data/vocab/metric.ttl and data/vocab/unit.ttl and both were accepted as genuinely missing; the metric was added to data/vocab/metric.ttl in this pass. This is a comparator cell. The printed method name was checked against the card and against the label column of the summary.csv row behind it, and the comparator configuration carries no model record because no primary source for the method was retrieved.

Field: attributes.printed_value

Source artifact SHA-256: 38479d8e4c6e46ed7e1ec4ece0dd02c358ae46d73d56790c44be813201fd9b98

Hash scope: Hash scope not separately documented; inspect source record

Archive member: artifacts/hf_README.md.gz

Extraction artifact SHA-256: 38479d8e4c6e46ed7e1ec4ece0dd02c358ae46d73d56790c44be813201fd9b98

Extraction artifact

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

4 source records and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: rfd4-proteina-20261115-result-boltzgen-dna-binders

areas
proteins-complexes; protein-structure; molecular-interactions
contexts
research
aggregation
Arithmetic mean of target-level rates
conditioning
Fixed-DNA conditioning: the DNA sequence and its coordinates are both specified.
coverage
eligible: 600; note: 3 targets times 200 original attempts; scored: 600; unit: design attempts
denominator
600
limitations
Printed to two decimals with no interval; the card prints no uncertainty for any cell.; The equivalent row in summary.csv is printed per 10 GPU-minutes; the per-GPU-hour value is the card's own printed figure.
metric
passing-design-clusters-per-accelerator-hour
metric direction
higher
metric qualifier
mean over targets of structurally distinct computational passers per GPU-hour, counting generation and common AlphaFold3 evaluation time, with 200 original attempts per target and failed attempts included; passing designs clustered within each model, protocol and target by Foldseek at TM 0.5; pass criterion: AF3 minimum interface PAE at most 1.5 A and DNA-target-aligned binder backbone RMSD at most 8 A, aligning on all target DNA atoms in both the fixed and unfixed modes
missing metadata
hardware: note: GPU-hours are reported without naming the GPU model used for the benchmark runs.; reason: unreported; numerator: note: The table prints the mean rate, not the passing-cluster count behind it; per-target rows are in benchmarks/source_data.csv, which was not retrieved.; reason: unreported; seed: note: No seeds are published for the generation runs.; reason: unreported; uncertainty: note: The table prints point values with no interval or spread.; reason: unreported
numeric value
9.46
printed source cell
BoltzGen (fixed DNA) · 9.46
printed value
9.46
review
artifact sha256: 38479d8e4c6e46ed7e1ec4ece0dd02c358ae46d73d56790c44be813201fd9b98; method: independent-cell-check; deterministic-table-parse; source-hash-verification; method note: Independent review pass. All thirteen pinned artifacts were re-retrieved on 2026-11-15 and every SHA-256 checked against its source record; the ten archived gzipped artifacts were each decompressed and hash-checked against the raw artifact_sha256, and each .gz against its archive_sha256. The eight table cells were then re-read by a reviewer script written for this pass (review/check_values.py), which finds the 'Measured design efficiency' section by its own heading, asserts the four printed header cells and the four row labels, requires each value to be printed to two decimals, splits the comparator cells on the printed middle dot, and does not import or reuse extract/parse_efficiency_table.py. Each stored printed_value, numeric_value, denominator, metric, unit, metric_direction and conditioning arm was compared against that parse and against the archived benchmarks/summary.csv, which prints the same quantity per 10 GPU-minutes: every card value equals its summary.csv unique_per_10_gpu_min times six at the printed precision, each row's attempts equals its targets times 200, and each cell's arm equals the dataset column of the summary.csv row behind it. The reviewer script also re-derived which summary.csv row each cell must come from, confirming that every RFD4 cell is the rfd4_proteina_high row and every comparator cell the highest non-RFD4 unique_per_10_gpu_min in that panel and criterion. All eight cells were in addition read by eye against the card, column assignment included, since there are only eight. Results are in review/value-check.tsv: 8 of 8 in_cell, no mismatch. npm run batch -- check-values was not run and does not apply: it reads JATS XML tables or a pdftotext -layout text layer, and this table is a Markdown table in a model card.; note: Every null and every missing_metadata entry on this result was checked against the sources and is genuinely unstated: no GPU model is named for the benchmark runs, no seed is published, and the passing-cluster count behind the rate is in benchmarks/source_data.csv, which this batch did not retrieve, so numerator is absent. No uncertainty is printed for any cell. The metric and unit concepts passing-design-clusters-per-accelerator-hour and count-per-hour were reviewed against data/vocab/metric.ttl and data/vocab/unit.ttl and both were accepted as genuinely missing; the metric was added to data/vocab/metric.ttl in this pass. This is a comparator cell. The printed method name was checked against the card and against the label column of the summary.csv row behind it, and the comparator configuration carries no model record because no primary source for the method was retrieved.; retrieval url: https://huggingface.co/nvidia/RFD4-Proteina/raw/621b785abf9d61dd24f4b81ae4dbd1330728f030/README.md; reviewed at: 2026-11-15T00:00:00Z; reviewer: claude; reviewer note: Claude (Opus 5) review agent, independent of the agent that extracted this batch; no human review claimed. Transcription was checked against the pinned sources; nothing was reproduced and no value was recomputed.
scope note
Computational passers per GPU-hour, not experimentally active designs. The card states that computational passers do not establish experimental activity.
source cells
Hugging Face model card, 'Measured design efficiency' table, row 'DNA binders', column 'Best other method in this comparison' (printed as 'BoltzGen (fixed DNA) · 9.46'); benchmarks/summary.csv, panel 'dna', label 'BoltzGen (fixed)', criterion 'dna', dataset 'dna_fixed', column unique_per_10_gpu_min = 1.5767861157347076
source locator
Hugging Face model card, 'Measured design efficiency' table, row 'DNA binders', column 'Best other method in this comparison' (printed as 'BoltzGen (fixed DNA) · 9.46')
unit
count-per-hour
unit detail
structurally distinct computational passers per GPU-hour, counting generation time and common AlphaFold3 evaluation time
Related records

Suggest a correction