HyInv report: baseline diversity mismatch
Baseline diversity mismatch of HyInv technical report inverse-folding comparison
The ten baselines are single published settings at one temperature each, not tuned to the panels, so comparisons are at comparable rather than exactly equal diversity. Of the 70 comparisons (ten baselines in seven panel-predictor columns) all 70 point estimates favour HyInv and 61 have an interval that excludes zero; the nine that include zero are ProteinMPNN (TS500, TS50), ProteinMPNN grouped (TS50), ProteinZero V2 ESM1 (CATH4.3 test, TS500, TS50), ProteinZero V2 ESM2 (CATH4.3 test, TS50) and LigandMPNN (TS50). PiFold, CMLM, LM-Design and DPLM-2 3B are single points at much lower diversity than HyInv, so their differences are not at matched diversity. ProteinMPNN is the only baseline with a temperature sweep, and it is the only baseline for which the report reads an equal-diversity difference. HyInv's diversity is above ProteinMPNN with a clear interval on CATH4.3 test and CATH4.2 test and above ProteinZero V2 (ESM2) on the same two panels, but 0.015 below ProteinZero V2 (ESM2) on RecentPDB; at the development curve's slope of 0.07 scTM per unit of diversity the report puts the largest offset (0.016) at a worth of at most 0.0011 scTM.
Descriptive fact read from the pinned sources at the locator given. Not a measurement on a benchmark panel.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
6 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Claimed field baseline_diversity_mismatch Context-only references | HyInv Technical Report (Hyra Team, 2026-10-14) Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Version: Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by extract/parse.py with every row label and column count asserted. |
| Source location Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Context-only references | HyInv Technical Report (Hyra Team, 2026-10-14) Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Version: Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by extract/parse.py with every row label and column count asserted. |
| Claimed value The ten baselines are single published settings at one temperature each, not tuned to the panels, so comparisons are at comparable rather than exactly equal diversity. Of the 70 comparisons (ten baselines in seven panel-predictor columns) all 70 point estimates favour HyInv and 61 have an interval that excludes zero; the nine that include zero are ProteinMPNN (TS500, TS50), ProteinMPNN grouped (TS50), ProteinZero V2 ESM1 (CATH4.3 test, TS500, TS50), ProteinZero V2 ESM2 (CATH4.3 test, TS50) and LigandMPNN (TS50). PiFold, CMLM, LM-Design and DPLM-2 3B are single points at much lower diversity than HyInv, so their differences are not at matched diversity. ProteinMPNN is the only baseline with a temperature sweep, and it is the only baseline for which the report reads an equal-diversity difference. HyInv's diversity is above ProteinMPNN with a clear interval on CATH4.3 test and CATH4.2 test and above ProteinZero V2 (ESM2) on the same two panels, but 0.015 below ProteinZero V2 (ESM2) on RecentPDB; at the development curve's slope of 0.07 scTM per unit of diversity the report puts the largest offset (0.016) at a worth of at most 0.0011 scTM. Context-only references | HyInv Technical Report (Hyra Team, 2026-10-14) Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Version: Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by extract/parse.py with every row label and column count asserted. |
| Description Descriptive fact read from the pinned sources at the locator given. Not a measurement on a benchmark panel. Context-only references | HyInv Technical Report (Hyra Team, 2026-10-14) Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Version: Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by extract/parse.py with every row label and column count asserted. |
| Relationship: subject hyinv-20261106-benchmark-report-comparison Context-only references | HyInv Technical Report (Hyra Team, 2026-10-14) Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Version: Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by extract/parse.py with every row label and column count asserted. |
| Name HyInv report: baseline diversity mismatch Context-only references | HyInv Technical Report (Hyra Team, 2026-10-14) Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2 Version: Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by extract/parse.py with every row label and column count asserted. |
Sources and history
Release 2026-11-21-c8bea25e5abb · Record review: source checked
1 source record and release history
- HyInv Technical Report (Hyra Team, 2026-10-14) · Original source · Dated 2026-10-14; retrieved at commit 0dcc533cc9f8f76c90ecbfe944df816b6672328a (21 pages)
Technical metadata and extraction receipts
Stable ID: hyinv-20261106-claim-benchmark-baseline-diversity-mismatch
- areas
- proteins-complexes; protein-structure
- contexts
- research
- field
- baseline_diversity_mismatch
- review
- artifact sha256: 0f10b390be399c341d1f7497c5a3ca0ac857ce78172c4732bdc86a9c66e46419; method: independent-cell-check; deterministic-table-parse; pdf-text-parse; source-hash-verification; method note: Independent review pass. The pinned PDF was re-downloaded at the pinned commit and its SHA-256 checked against the source record; pdftotext -layout reproduced the archived text layer byte for byte. npm run batch -- check-values was run over the whole batch against that text layer (value-check.tsv). The 170 cells of Tables 6, 7, 8 and 9 and the 290 cells of the Figure 14 and Figure 15 matrices were then re-read by a reviewer script that finds each row by its printed label and splits the printed cells by position, written for this pass without using the extractor's parser; every printed value and column matched. Column assignment in Figure 14 was confirmed against the labelled columns of Table 2 and against Section 5.2's prose figures; Figure 15's was confirmed by the coordinator reading the rotated column headers on the rendered page 17, which match Figure 14's order. Table 1, Figure 4a and the eleven prose values were read by hand against the text layer. Each claim was re-read against the passage its locator names.; retrieval url: https://raw.githubusercontent.com/Tencent-Hunyuan/Hyra-results/0dcc533cc9f8f76c90ecbfe944df816b6672328a/AI4Science/hyinv/HyInv_Technical_Report.pdf; reviewed at: 2026-11-06T00:00:00Z; reviewer: claude; reviewer note: Claude research agent; no human review claimed. Reviewed by a different agent from the one that extracted the batch. Transcription checked against the pinned source; nothing was reproduced and no value was recomputed.
- source locator
- Appendix D.1, D.2 and D.3; Appendix A 'Scores at equal diversity'; Section 5.2
- value
- The ten baselines are single published settings at one temperature each, not tuned to the panels, so comparisons are at comparable rather than exactly equal diversity. Of the 70 comparisons (ten baselines in seven panel-predictor columns) all 70 point estimates favour HyInv and 61 have an interval that excludes zero; the nine that include zero are ProteinMPNN (TS500, TS50), ProteinMPNN grouped (TS50), ProteinZero V2 ESM1 (CATH4.3 test, TS500, TS50), ProteinZero V2 ESM2 (CATH4.3 test, TS50) and LigandMPNN (TS50). PiFold, CMLM, LM-Design and DPLM-2 3B are single points at much lower diversity than HyInv, so their differences are not at matched diversity. ProteinMPNN is the only baseline with a temperature sweep, and it is the only baseline for which the report reads an equal-diversity difference. HyInv's diversity is above ProteinMPNN with a clear interval on CATH4.3 test and CATH4.2 test and above ProteinZero V2 (ESM2) on the same two panels, but 0.015 below ProteinZero V2 (ESM2) on RecentPDB; at the development curve's slope of 0.07 scTM per unit of diversity the report puts the largest offset (0.016) at a worth of at most 0.0011 scTM.