What the evidence shows
Two preprints report formula-free retrieval of the true structure from mass-matched PubChem candidates for MassSpecGym spectra, and their values are not directly comparable.
In the MSAlign preprint, MSAlign ranks first among the formula-free methods on every split the paper reports, but its top-1 recall falls from about half on the formula split to under a fifth on the MCES split. On the MassSpecGym formula split, Recall@1 was 47.5 for MSAlign and 58.2 with score fusion, against 42.7 for Emb-Cos, 13.9 for JESTR and 4.7 for DeepSet, with standard deviations of 0.5 to 3.4 over three random splitting seeds. On the MCES split the order is the same but the values are much lower: 19.0, 30.8, 12.6, 7.9 and 1.9; the table caption describes three splits there too, but the methods describe two swapped variants.
Recall@20 reaches 87.0 for MSAlign and 92.8 with score fusion on the formula split, and 64.9 and 82.0 on MCES. An earlier version of the same paper reports Recall@1 of 53.8 for MSAlign and 41.2 for Emb-Cos on the formula split, so the two versions differ and are not independent measurements. These MSAlign values assume the true structure is among 256 mass-matched candidates and were scored by the MSAlign authors.
MassSpecGym in the Wild, whose authors include the developers of MIST, MIST-CF and MassSpecGym, reports two formula-free configurations that it does not flag for formula leakage, on the MassSpecGym test fold with up to 256 mass-matched candidates per spectrum. MIST given the top formula predicted by MIST-CF found the true structure at rank 1 for 26.64 percent of test spectra and within the top 20 for 54.90 percent, against 0.43 and 7.99 percent for random ranking. Its candidate pool and split differ from MSAlign's, and the MIST score depends on the formula annotator as well as the retriever.
All of these values are reported by the authors of preprints, and none measures open-world identification, because the true structure is always among the candidates.
- You bring
- MS/MS spectra with metadata sufficient to derive neutral molecular mass
- A frozen set of candidate molecular structures; the true identity must be present for the measured retrieval endpoint
- You want
- Split-specific Recall@1 and Recall@20 evidence for six exact configurations, plus candidate-pool and domain-shift limits.
- Evidence in this release
- 8 comparisons shown · 9 tools · 13 configurations
- Kept up to date
- Evidence last checked 7 October 2026 · re-checked monthly, next due 7 November 2026
For "Shortlist molecular identities from tandem mass spectra", on MSAlign molecular retrieval MassSpecGym formula split: no formula, R@1: MassSpecGym formula split retrieval with no formula, Top-1 accuracy (retrieval) (Top-1 accuracy, a proxy protocol), MSAlign scored 53.8%, reported by msalign: Primary paper PDF.
Best reported valueMSAlignmsalign: Primary paper PDF
Top-1 accuracy53.8%MSAlign molecular retrieval MassSpecGym formula split: no formula, R@1: MassSpecGym formula split retrieval with no formula, Top-1 accuracy (retrieval)
Baseline on this protocolNone scoredNo margin is computed
Ranked tools65 from independent studies
6 tools have a value on this protocol; 5 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.
Ranking
Ranked on MSAlign molecular retrieval MassSpecGym formula split: no formula, R@1: MassSpecGym formula split retrieval with no formula, Top-1 accuracy (retrieval) (Top-1 accuracy, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.
6 tools, from 1 source. 5 of 6 results (83%) come from an independent study; 1 is reported by the tool's developers. The top-ranked result is reported by the tool's developers.
5 tools are evaluated elsewhere in this use case but not on this protocol, so not ranked
Comparisons
Unified model comparison for this use case. Compare tools against protocol baselines in a single view.
Proxy evidenceDeveloper-reported: Reported by the developers of the evaluated tool.Independent study: Reported in a study whose authors did not develop the evaluated tool.
Recall@1 of the true molecular structure among 256 mass-matched candidate structures on the MassSpecGym formula split, with no molecular formula supplied.
Sorted by Top-1 accuracy (higher is better). The best value in each column is highlighted.
| Tool | Top-1 accuracy (retrieval) |
|---|
| 53.8% (best in column) |
|---|
| 41.2% |
|---|
| 29.5% |
|---|
| 22.2% |
|---|
| 20.0% |
|---|
| 16.9% |
|---|
Caveats and method
- The correct structure must be in a 256-candidate mass-filtered pool. Scores do not transfer automatically to a larger database, a different pool-construction procedure or a missing true structure.
- MCES and formula splits impose different distribution shifts. The source authors favour formula splits for their deployment interpretation; that choice does not prove that formula-split performance represents every intended application.
- Reported methods are the paper implementations: DeepSets training was extended to 50 epochs. These are not interchangeable with original model-paper scores or current checkpoints.
- Exact test denominators, split-manifest hashes, checkpoint hashes and evaluator revisions remain unextracted or unreported in the catalogue. No uncertainty values or new independent reproduction are established.
- Candidate recall is not an authenticated metabolite identification, a clinical diagnosis or a calibrated probability. Human domain review remains outstanding.
Protocol: MSAlign molecular retrieval MassSpecGym formula split: no formula, R@1: MassSpecGym formula split retrieval with no formula
Primary benchmark evaluation shown (MSAlign molecular retrieval MassSpecGym formula split: no formula, R@1: MassSpecGym formula split retrieval with no formula). 7 additional study protocols (MSAlign v2 retrieval: massspecgym-mces-formula-free, MSAlign v2 retrieval: massspecgym-formula-formula-free, MSAlign v2 retrieval: spectraverse-formula-free and others) are cited in individual tool drawer rows and under Question, sources and review.
Not covered yet
- The correct structure must be in a 256-candidate mass-filtered pool. Scores do not transfer automatically to a larger database, a different pool-construction procedure or a missing true structure.
- MCES and formula splits impose different distribution shifts. The source authors favour formula splits for their deployment interpretation; that choice does not prove that formula-split performance represents every intended application.
- Reported methods are the paper implementations: DeepSets training was extended to 50 epochs. These are not interchangeable with original model-paper scores or current checkpoints.
- Exact test denominators, split-manifest hashes, checkpoint hashes and evaluator revisions remain unextracted or unreported in the catalogue. No uncertainty values or new independent reproduction are established.
- Candidate recall is not an authenticated metabolite identification, a clinical diagnosis or a calibrated probability. Human domain review remains outstanding.
- The four metric and split groups come from one paper and do not constitute independent replications.
- 30 September 2026 audit: v1 source record labels 2605.19752v1 but uses an unversioned URL that now serves v2; old source bytes/values must remain immutable. v1-specific URL returned HTTP406 during this audit.
- 30 September 2026 audit: For MCES, the v2 Table3 generic caption says three random splits, but Section5.1 explicitly uses two validation/test-swapped variants; the discrepancy is preserved.
- 30 September 2026 audit: Benchmark assumes the true structure is present among 256 neutral-mass-matched candidates. Unknown candidates, instrument-specific prospective performance and authenticated identification remain gaps.
Contribute evidence or propose a correction
How to cite
Shortlist molecular identities from tandem mass spectra. Rewire biological model benchmark database, release 2026-11-21-c8bea25e5abb. https://benchmarks.rewirebio.io/use-cases/mass-spectrum-molecule-shortlisting/
Release 2026-11-21-c8bea25e5abb. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.
The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence
Question, sources and review
Who this is for, setting and exclusions
Compare formula-free retrieval configurations within a named split and shortlist size, then validate the intended instrument, chemistry and candidate database before selecting a workflow.
- Who this is for
- Metabolomics researchers shortlisting candidate molecular identities for experimental confirmation
- Computational researchers choosing retrieval inputs, candidate pools and validation splits
- Biological setting
- MSAlign paper Table 3, MassSpecGym formula and MCES splits, formula-free inputs. Each source candidate pool contains 256 mass-matched PubChem structures. Formula and MCES splits remain distinct.
- Research setting
- Research only. Candidate retrieval is not a confirmed molecular identification or evidence of clinical diagnostic performance.
Outside this use case
- De novo molecular generation or cases where the true structure is absent from the candidate set
- Formula-conditioned retrieval, including MIST, FLARE and MSAlign+Filter
- Pooling split-specific results or transferring scores to different candidate database sizes
- Authenticated molecular identification and clinical diagnosis
Sources, review and downloads
Automated source review · 2026-09-30 · Codex source curation, independent worker cross-review and root integration review
Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.
Release 2026-11-21-c8bea25e5abb · question use-case-mass-spectrum-molecule-shortlisting
Use-case input digest bfbb7ab9303becad1e3cb395568c26f41e72ddb67cd1a89a06d9eaa0f76bd95f
Download questions and review metadata (JSON) (gzip) · Verify release checksums (gzip)