Relevance of MassSpecGym mass-based retrieval, Hit rate@20 (MassSpecGym in the Wild Table 2) to "Shortlist molecular identities from tandem mass spectra"

Stable ID: use-case-mapping-lit-otheromics-20261011-msgwild-mass-retrieval-hit-rate-20

field

links:assessed_by:lit-otheromics-20261011-msgwild-protocol-mass-retrieval-hit-rate-20

value

lit-otheromics-20261011-msgwild-protocol-mass-retrieval-hit-rate-20

relevance

proxy

endpoint

Hit rate@20 (mean over MassSpecGym test spectra, 99.9% bootstrap CI) for ranking the true structure among mass-matched PubChem candidates

rationale

Proxy, not direct. The mass-based track withholds the formula, which matches the use case's formula-free input and shortlist endpoint. MIST here takes the top-1 formula predicted from the spectrum by a MIST-CF retrained on the MassSpecGym training split only, and that formula is not used to filter candidates (Section 4.3), so the pipeline needs only the spectrum and precursor mass. This differs from the formula-conditioned MIST the use case excludes, which receives the true formula. Only two valid configurations are measured (random ranking and this MIST pipeline), and the candidate pool and split differ from the MSAlign settings the use case was built on.

constraints

Use only the two non-grey evaluations; the grey rows leak ground-truth or test-trained formula information and are excluded.; Treat MIST with the MIST-CF top-1 formula as a two-model pipeline (formula annotator plus formula-aware retriever). Do not transfer its score to MIST given a known formula, or to MIST-CF checkpoints trained on test spectra.; Do not combine with the formula-known (bonus) retrieval results of Table 3 or with MSAlign's split-specific pools.; Compare within this table only; corrected MassSpecGym v1.5 candidate canonicalization may change other published values.

limitations

Authors include the MIST, MIST-CF and MassSpecGym developers (author_reported).; Preprint (arXiv:2606.19624v1).; The true structure must be in the candidate set; not open-world identification.; Retrieval inherits the errors of the formula annotator; its formula accuracy is in Table 7, not extracted.; Table 8 reports a further non-grey variant (MIST-CF top-5 formulas, element-wise max), not extracted.; Values are percentages printed without a percent sign in Table 2.

citation locators

source id: lit-otheromics-20261011-msgwild-source-liu2026; locator: Table 2, column 'Hit rate@20'; Section 4.3; Appendix 'Confidence Intervals'

source locator

lit-otheromics-20261011-msgwild-source-liu2026: Table 2, column 'Hit rate@20'

headline metric

top-20-accuracy

reason

Add a relevance judgement for the MassSpecGym in the Wild mass-based retrieval protocol from the 2026-10-11 other-omics literature pass.

comparison group

massspecgym-in-the-wild-mass-retrieval

comparison title

MassSpecGym in the Wild, mass-based retrieval (Table 2)

stratum label

Hit rate@20

stratum order

2

excluded evaluations

id: lit-otheromics-20261011-msgwild-eval-random-ranking-among-candidates-filtered-by-the-ground-truth-formula-mass-retrieval-hit-rate-20; reason: Grey in the source: Grey: ground-truth formula leaks into the mass-based track. Not a valid formula-free result.; id: lit-otheromics-20261011-msgwild-eval-mist-retrieval-with-top-1-formula-from-mist-cf-trained-on-test-data-only-mass-retrieval-hit-rate-20; reason: Grey in the source: Grey: formula annotator trained on test spectra. Not a valid formula-free result.; id: lit-otheromics-20261011-msgwild-eval-mist-retrieval-with-top-1-formula-from-mist-cf-trained-on-train-and-test-data-mass-retrieval-hit-rate-20; reason: Grey in the source: Grey: formula annotator trained on test spectra. Not a valid formula-free result.; id: lit-otheromics-20261011-msgwild-eval-mist-retrieval-with-the-ground-truth-formula-mass-retrieval-hit-rate-20; reason: Grey in the source: Grey: ground-truth formula leaks into the mass-based track. Not a valid formula-free result.

reviewed evaluations

lit-otheromics-20261011-msgwild-eval-random-ranking-mass-retrieval-hit-rate-20; lit-otheromics-20261011-msgwild-eval-mist-retrieval-with-mist-cf-top-1-predicted-formula-mass-retrieval-hit-rate-20

review

method: automated-source-review; reviewer: claude; reviewer note: Claude review agent reviewer-ro-2, independent of the extractor lit-rna-otheromics; no human review claimed; reviewed at: 2026-10-12T07:44:06Z; note: Independent review of endpoint fit, relevance grade, grey-row exclusions and transfer limits against the use case and MassSpecGym in the Wild (arXiv:2606.19624v1) Table 2 and Section 4.3, with every Table 2 value checked against the source. Rationale, constraints and limitations are the reviewer's text (M8). No execution, reproduction, human scientific review or clinical validation.

pins

record id: lit-otheromics-20261011-msgwild-eval-mist-retrieval-with-mist-cf-top-1-predicted-formula-mass-retrieval-hit-rate-20; fields: status; links; attributes.origin; attributes.comparison; sha256: 52b8b0313657c03fc8f5de1787cace7eb8f21a2bbb678a94090e5d4f459d1a7f; record id: lit-otheromics-20261011-msgwild-eval-random-ranking-mass-retrieval-hit-rate-20; fields: status; links; attributes.origin; attributes.comparison; sha256: f5275452be6d3fc7b1bc12e9acd655e681bb0eb02e3a31f575766e805666ebe5; record id: lit-otheromics-20261011-msgwild-protocol-mass-retrieval-hit-rate-20; fields: status; name; description; facets; source_ids; links; attributes; sha256: 96424f2f890450415342a4b3eace43cb49d789306618331d0e5bf3fc0b173487; record id: lit-otheromics-20261011-msgwild-result-mist-retrieval-with-mist-cf-top-1-predicted-formula-mass-retrieval-hit-rate-20; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4225dbe5aeef04f601973d7a59660ce45fd5eb079d8a203c7fc6c03002150286; record id: lit-otheromics-20261011-msgwild-result-random-ranking-mass-retrieval-hit-rate-20; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 67d11856ca69d123c99f88e5ea7b22ae359bf75f1125979800778f2ad678791f; record id: lit-otheromics-20261011-msgwild-source-liu2026; fields: status; attributes.artifact_sha256; sha256: 16f0230179c202cf334b460914c18740aebd1900243b7139ecce372f36dc8d62; record id: use-case-mass-spectrum-molecule-shortlisting; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 6dc5b0ef9fea31717195b4d9fa5bf871b774a09b18f0f066d499e2abe1f293de

• subject: Shortlist molecular identities from tandem mass spectra