rewire.itbenchmarks
← Back to results
Benchmark

MassSpecGym

MassSpecGym separates spectrum-to-structure generation, candidate retrieval and structure-to-spectrum simulation.

Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes

24 evaluations · 116 results

Overview

Datasets

The released MassSpecGym MS/MS/molecule dataset, with task-specific inputs and candidate sets.

Metrics

Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.

Allowed inputs

Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.

Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
Evaluation procedure diagram
How it worksEvaluation procedure
Evaluation procedure1. Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.. Then: 2. Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.. Then: 3. Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.Evaluation procedure1. Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.. Then: 2. Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.. Then: 3. Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.Evaluation procedure1. Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.. Then: 2. Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.. Then: 3. Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.

Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.

Sources (5)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py; massspecgym primary benchmark evidence · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Source reviewed · Automated source review, 2026-09-16. All specifications and missing details

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

MassSpecGym · main · Table 2

Top-1 accuracy (fraction) · Higher values are better.

MassSpecGym · main (MassSpecGym De novo molecule generation) · MassSpecGym · main

Evidence origin: Author-reported evaluation.

MassSpecGym: A benchmark for the discovery and identification of molecules · Table 2: Top-1 accuracy, main
  • Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.
  • Zero generation accuracy is a reported result, not missingness.
  • No checkpoint revision inferred from method name.
  • Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.
  • Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.
Comparison details and limitations

Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split

Automated source review: 2026-09-17. Numerical source review does not establish independent reproduction.

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Showing 3 of 3 matching rows.

Tested configuration
00.250.50.751
Reported score
  1. Random chemical generation (main)0
  2. SMILES Transformer (main)0
  3. SELFIES Transformer (main)0

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

The released MassSpecGym MS/MS/molecule dataset, with task-specific inputs and candidate sets. MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold. Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores. Maximum common edge subgraph (MCES) clustering keeps molecules connected by a bond-edit distance below 10 in the same fold. The split additionally balances instrument, collision-energy, adduct and molecule-frequency metadata; this is stronger than simply separating 2D InChIKeys.

Sources (5)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py; massspecgym primary benchmark evidence · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5; Section 3.4; Supplementary Information 2.5; Tables 2–4

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Tasks

Protocols

These source-backed links do not make different protocols or scores interchangeable.

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 12 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

Baseline status by linked protocol

Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums

Coverage is derived from release 2026-09-29-06401fd5b220. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run this benchmark

Choose a concrete protocol before running an evaluation. Its inputs, split and scoring rules determine which results can be compared.

Load MassSpecGym and evaluate a model

Install the package and load the benchmark dataset that the scores on this page are measured over.

Generate predictions and evaluate them. This recipe does not establish reproduction of a particular published score.

Dataset access
Downloaded by the package on first use.
Model and weights
None required.
Licences
Project licence: MIT. Upstream data licences are separate and unreported here.
Software
Python with the massspecgym package from PyPI.
Hardware
Not stated in the cited section. Several of these steps expect a GPU.
Required inputs and expected outputs

Inputs

  • A model for one of the benchmark's spectrum tasks.

Outputs

  • The benchmark dataset and the scores its evaluation produces.

Execution steps

  1. 1. Install (Command line)

    Source reviewed; these instructions have not been executed by rewire.

    pip install massspecgym
    MassSpecGym: repository README · README.md at f259fe37, Installation, lines 43-43

  2. 2. Load the dataset (Python)

    Source reviewed; these instructions have not been executed by rewire.

    from massspecgym.utils import load_massspecgym
df = load_massspecgym()
    MassSpecGym: repository README · README.md at f259fe37, Getting started with MassSpecGym, lines 76-77

Use your own model

Run your model locally and return predictions keyed by the input IDs. The evaluator supplies biological inputs without test labels and owns scoring. This interface is not a sandbox for model code.

Pass your existing prediction function into this adapter. Its output direction must match the selected protocol.

class MyModelAdapter:
    def __init__(self, score):
        self.score = score

    def predict(self, inputs):
        return {row["id"]: float(self.score(row)) for row in inputs}

# adapter = MyModelAdapter(your_prediction_function)
# report = rewirebench.run(prepared, adapter, output="runs/my-model")

Alternatively, generate a keyed prediction file in your existing model environment and use the score-only recipe. Your model code and weights do not need to be shared.

MassSpecGym: repository README · README.md at f259fe37
Scope and limitations
  • Quoted from the project's README and not executed by rewire, so the commands are evidence of what the project documents rather than a verified run.
  • The project may have changed since the pinned commit.
  • The tasks use different metrics and are scored separately.

Contribute a result for review. The library can submit an exported evaluation for private review when intake is open. Check the contribution page for access and sign-in.

Original repository instructions

Official run instructions

Run the official CPU DeepSets retrieval tutorial: train a small spectrum-to-fingerprint model, then evaluate retrieval on the provided test split.

Checked against the official instructions on 2026-09-17. These commands have not been executed by rewire. Running them does not automatically reproduce the published scores.

Before you start

  • Conda and Git; the official README recommends a Python 3.11 environment.
  • Internet access for MassSpecGym dataset/candidate downloads from Hugging Face. The dataset loader downloads data as needed.

  1. 1. Check out the reviewed repository

    Repository checkout wrapper: the detached revision selects the exact official source inspected for this guide.

    git clone https://github.com/pluskal-lab/MassSpecGym.git
cd MassSpecGym
git checkout --detach f259fe3780d5bd227fc6ece36ce6f397c2eef716
    pluskal-lab/MassSpecGym / README.md · Pinned repository revision; README.md

  2. 2. Install into the recommended environment

    The README package-install command is unversioned. Record the installed version; the pinned documentation commit is not an installed-package lock.

    conda create -n massspecgym python==3.11
conda activate massspecgym
pip install massspecgym
    pluskal-lab/MassSpecGym / README.md · README.md lines 38–51

  3. 3. Run the documented CPU training and test example

    The four official Python blocks are combined in a local script. A standard main guard is added around execution for the documented four data workers; model, CPU trainer, one device, five epochs and batch size 32 are unchanged.

    cat > massspecgym_retrieval_example.py <<'PY'
import torch
import torch.nn as nn
import pytorch_lightning as pl
from pytorch_lightning import Trainer

from massspecgym.data import RetrievalDataset, MassSpecDataModule
from massspecgym.data.transforms import SpecTokenizer, MolFingerprinter
from massspecgym.models.base import Stage
from massspecgym.models.retrieval.base import RetrievalMassSpecGymModel

class MyDeepSetsRetrievalModel(RetrievalMassSpecGymModel):
    def __init__(
        self,
        hidden_channels: int = 128,
        out_channels: int = 4096,  # fingerprint size
        *args,
        **kwargs
    ):
        """Implement your architecture."""
        super().__init__(*args, **kwargs)

        self.phi = nn.Sequential(
            nn.Linear(2, hidden_channels),
            nn.ReLU(),
            nn.Linear(hidden_channels, hidden_channels),
            nn.ReLU(),
        )
        self.rho = nn.Sequential(
            nn.Linear(hidden_channels, hidden_channels),
            nn.ReLU(),
            nn.Linear(hidden_channels, out_channels),
            nn.Sigmoid()
        )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """Implement your prediction logic."""
        x = self.phi(x)
        x = x.sum(dim=-2)  # sum over peaks
        x = self.rho(x)
        return x

    def step(
        self, batch: dict, stage: Stage
    ) -> tuple[torch.Tensor, torch.Tensor]:
        """Implement your custom logic of using predictions for training and inference."""
        # Unpack inputs
        x = batch["spec"]  # input spectra
        fp_true = batch["mol"]  # true fingerprints
        cands = batch["candidates"]  # candidate fingerprints concatenated for a batch
        batch_ptr = batch["batch_ptr"]  # number of candidates per sample in a batch

        # Predict fingerprint
        fp_pred = self.forward(x)

        # Calculate loss
        loss = nn.functional.mse_loss(fp_true, fp_pred)

        # Calculate final similarity scores between predicted fingerprints and retrieval candidates
        fp_pred_repeated = fp_pred.repeat_interleave(batch_ptr, dim=0)
        scores = nn.functional.cosine_similarity(fp_pred_repeated, cands)

        return dict(loss=loss, scores=scores)

if __name__ == '__main__':
    # Init hyperparameters
    n_peaks = 60
    fp_size = 4096
    batch_size = 32

    # Load dataset
    dataset = RetrievalDataset(
        spec_transform=SpecTokenizer(n_peaks=n_peaks),
        mol_transform=MolFingerprinter(fp_size=fp_size),
    )

    # Init data module
    data_module = MassSpecDataModule(
        dataset=dataset,
        batch_size=batch_size,
        num_workers=4
    )

    # Init model
    model = MyDeepSetsRetrievalModel(out_channels=fp_size)

    # Init trainer
    trainer = Trainer(accelerator="cpu", devices=1, max_epochs=5)

    # Train
    trainer.fit(model, datamodule=data_module)

    # Test
    trainer.test(model, datamodule=data_module)
PY
python massspecgym_retrieval_example.py
    pluskal-lab/MassSpecGym / README.md · README.md lines 111–218

Expected outputs

  • Lightning training/validation logs followed by retrieval test metrics from trainer.test on the official data module.

Scope and limitations

  • This is the README tutorial model, not a claim to reproduce a published leaderboard checkpoint.
  • The example explicitly runs on CPU; runtime, memory demand and storage capacity are not stated.
  • The PyPI installation, transitive dependencies and downloaded data are not hash-locked. The Python script wrapper has been source-reviewed only, not executed.
Strengths, limitations and unresolved questions

Strengths and limitations

Strengths supported by sources

Limitations and conditions

  • Chemical-formula-assisted tasks provide extra input information and must remain separate from unassisted tasks. The MCES split constrains structural similarity but cannot establish independence from every external pretraining corpus.
    Sourcesmassspecgym primary benchmark evidence · Section 3.4; Supplementary Information 2.5; Tables 2–4
Profile review details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Stable record: discovery-benchmark-massspecgym

Specifications

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
PropertyDescription and evidence
DatasetsThe released MassSpecGym MS/MS/molecule dataset, with task-specific inputs and candidate sets.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
SplitsMCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.
Sourcesmassspecgym primary benchmark evidence · Section 3.4; Supplementary Information 2.5
MetricsTask evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
BaselinesThe README illustrates a DeepSets-style spectrum-to-fingerprint retrieval baseline.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
Leakage controlsMaximum common edge subgraph (MCES) clustering keeps molecules connected by a bond-edit distance below 10 in the same fold. The split additionally balances instrument, collision-energy, adduct and molecule-frequency metadata; this is stronger than simply separating 2D InChIKeys.
Sourcesmassspecgym primary benchmark evidence · Section 3.4; Supplementary Information 2.5; Tables 2–4
UncertaintyTables 2–4 report 99.9% bootstrap confidence intervals using 20,000 resamples. These intervals summarize test-example sampling, not variation across independently retrained models.
Sourcesmassspecgym primary benchmark evidence · Section 3.4; Supplementary Information 2.5; Tables 2–4
Entity typeSmall-molecule MS/MS benchmark with three task directions.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
OrganismsMolecule identity rather than organism classification defines these tasks. · Not applicable
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
AssaysTandem mass spectra paired with molecular structures.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
Allowed inputsSpectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
AdaptationSupervised train/validation/test learning; pretrained or new models use the task-specific interfaces.
Sources (4)pluskal-lab/MassSpecGym official source; pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py; pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py; pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py · Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes
Applicable tests and references

Applicability is distinct from a completed evaluation.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resourceVersionReference
MassSpecGym: A benchmark for the discovery and identification of molecules2410.23326v1Read source
Historical gaps recorded on 2026-09-17

The catalogue now holds 116 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

  • Independent batch review before import; preserve existing observation identities.
Search and extraction details

complete comparison tables extracted pending publication review

Searches

  • MassSpecGym 2410.23326 benchmark results Table 1 Table 2

Evidence locations

  • Tables2–4; task definitions and supplementary split table

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

74 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statementOriginal source and locationReview and provenance
Diagram caption
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Individual claims		pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T20:42:47.661606+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 5d2d61d7ac3c6503b3869d5cec687e7e82d438900df3d01c6f7cf4c340e5a004

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram caption
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Individual claims		pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T20:42:47.889112+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 2ae08f58bd6db0e00430b54c078b45ae91b9252aac02ee358fef9d8ba1b3f81f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram caption
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Individual claims		pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T20:42:48.138101+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 5208cc352c144e24ea8249124483856aeb1e9821c0cea142d6a907ac248b6295

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram caption
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Individual claims		massspecgym primary benchmark evidence

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2410.23326v1
Retrieved: 2026-09-16T21:04:58.775040+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 82176d50e8947c8b9baa2a0d91493f5680c0e4c7e25a2266ff7879f48a58c40c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram caption
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Individual claims		pluskal-lab/MassSpecGym official source

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T10:30:23.954980+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 08bf3607e6e2e5462b81eac85d0e71d9d23ce1c9bf1a370c9d1079ecd60ee2d8

Hash scope: Hash scope not separately documented; inspect source record

Diagram steps
  • Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.
  • Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.
  • Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.
Individual claims		pluskal-lab/MassSpecGym massspecgym/models/de_novo/base.py

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T20:42:47.661606+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 5d2d61d7ac3c6503b3869d5cec687e7e82d438900df3d01c6f7cf4c340e5a004

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.
  • Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.
  • Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.
Individual claims		pluskal-lab/MassSpecGym massspecgym/models/retrieval/base.py

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T20:42:47.889112+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 2ae08f58bd6db0e00430b54c078b45ae91b9252aac02ee358fef9d8ba1b3f81f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.
  • Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.
  • Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.
Individual claims		pluskal-lab/MassSpecGym massspecgym/models/simulation/base.py

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T20:42:48.138101+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 5208cc352c144e24ea8249124483856aeb1e9821c0cea142d6a907ac248b6295

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.
  • Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.
  • Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.
Individual claims		massspecgym primary benchmark evidence

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2410.23326v1
Retrieved: 2026-09-16T21:04:58.775040+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 82176d50e8947c8b9baa2a0d91493f5680c0e4c7e25a2266ff7879f48a58c40c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Allowed inputs: Spectrum-to-molecule, spectrum-plus-candidates, or molecule-to-spectrum inputs depend on the selected task.
  • Splits: MCES molecular clusters are grouped into fixed training, validation and test folds, stratified by acquisition metadata. Cross-fold molecular bond-edit distance is at least 10; all spectra follow the assigned molecular fold.
  • Metrics: Task evaluators distinguish molecular exact match/structural similarity, candidate-retrieval hit rate and spectrum similarity. These are separate readouts, not interchangeable scores.
Individual claims		pluskal-lab/MassSpecGym official source

Original source ↗

Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; Section 3.4; Supplementary Information 2.5

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f259fe3780d5bd227fc6ece36ce6f397c2eef716
Retrieved: 2026-09-16T10:30:23.954980+00:00

source checked

automated source review · 2026-09-16

Audit details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 08bf3607e6e2e5462b81eac85d0e71d9d23ce1c9bf1a370c9d1079ecd60ee2d8

Hash scope: Hash scope not separately documented; inspect source record

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: discovered

8 source records and release historyDownload this release
Technical metadata and extraction receipts

Stable ID: discovery-benchmark-massspecgym

areas
metabolomics
entity level
suite
scope note
Specialist molecular or omics evaluation; protocol details require review before numerical comparison.
task
Molecular identification from tandem mass spectra
version
Not reported
comparison panels
id: part2-massspecgym-table2-eaf9691d7e; title: MassSpecGym · main · Table 2; protocol id: paper-protocol-9e2c05cdecc9a3241c; dataset id: paper-dataset-c25f9f3bb2961b23d5; metric: Top-1 accuracy; unit: fraction; direction: higher; result ids: paper-result-cdb00f17075ddae1d4; paper-result-001ce58c5292191491; paper-result-114b6658eaef67bace; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-1 accuracy, main; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-09fb9292ad; title: MassSpecGym · main · Table 2; protocol id: paper-protocol-9e2c05cdecc9a3241c; dataset id: paper-dataset-c25f9f3bb2961b23d5; metric: Top-1 MCES; unit: edge-edit distance; direction: lower; result ids: paper-result-5e7fd1597e90064dde; paper-result-5dbec6173d09a309db; paper-result-39e03e2651a3119076; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-1 MCES, main; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-f34201e2e4; title: MassSpecGym · main · Table 2; protocol id: paper-protocol-9e2c05cdecc9a3241c; dataset id: paper-dataset-c25f9f3bb2961b23d5; metric: Top-1 Tanimoto; unit: dimensionless; direction: higher; result ids: paper-result-fe122a4b95cfa19aec; paper-result-d8b7f603a24a1acf17; paper-result-a8a01911a2a38153bb; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-1 Tanimoto, main; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-363b3d3145; title: MassSpecGym · main · Table 2; protocol id: paper-protocol-9e2c05cdecc9a3241c; dataset id: paper-dataset-c25f9f3bb2961b23d5; metric: Top-10 accuracy; unit: fraction; direction: higher; result ids: paper-result-bc4465183b3bdd05e9; paper-result-b11072597f662ee8a3; paper-result-26d9400143fc2fd637; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-10 accuracy, main; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-9a67349dfe; title: MassSpecGym · main · Table 2; protocol id: paper-protocol-9e2c05cdecc9a3241c; dataset id: paper-dataset-c25f9f3bb2961b23d5; metric: Top-10 MCES; unit: edge-edit distance; direction: lower; result ids: paper-result-c4d079d645b148475c; paper-result-ef245bf2ce8115f850; paper-result-198ffa156afdd9aafa; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-10 MCES, main; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-8afdec59d8; title: MassSpecGym · main · Table 2; protocol id: paper-protocol-9e2c05cdecc9a3241c; dataset id: paper-dataset-c25f9f3bb2961b23d5; metric: Top-10 Tanimoto; unit: dimensionless; direction: higher; result ids: paper-result-c8b08ae7e71a836a51; paper-result-ddf10b5e3e4f655b34; paper-result-724980ddce45409480; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-10 Tanimoto, main; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-c5487be6c0; title: MassSpecGym · formula · Table 2; protocol id: paper-protocol-5d42ac5481b1c071e4; dataset id: paper-dataset-3f9a5b3d0316153887; metric: Top-1 accuracy; unit: fraction; direction: higher; result ids: paper-result-cb2bbb66e78675cc68; paper-result-cedb7d744e7a1a1270; paper-result-52e51c1dd59d05f572; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-1 accuracy, formula; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-eef6fb998a; title: MassSpecGym · formula · Table 2; protocol id: paper-protocol-5d42ac5481b1c071e4; dataset id: paper-dataset-3f9a5b3d0316153887; metric: Top-1 MCES; unit: edge-edit distance; direction: lower; result ids: paper-result-a3823cbc55713c1741; paper-result-f9e0659ed6de1857f1; paper-result-38f0c179e79c4c9dac; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-1 MCES, formula; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-78affa9e01; title: MassSpecGym · formula · Table 2; protocol id: paper-protocol-5d42ac5481b1c071e4; dataset id: paper-dataset-3f9a5b3d0316153887; metric: Top-1 Tanimoto; unit: dimensionless; direction: higher; result ids: paper-result-0ac9b7767e0fd6ca67; paper-result-7f9cce726b4019f851; paper-result-96030a32c14aaf11c3; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-1 Tanimoto, formula; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-daa1fccabf; title: MassSpecGym · formula · Table 2; protocol id: paper-protocol-5d42ac5481b1c071e4; dataset id: paper-dataset-3f9a5b3d0316153887; metric: Top-10 accuracy; unit: fraction; direction: higher; result ids: paper-result-57c8c3eaa6c96fd115; paper-result-252d625fb1507235e2; paper-result-23eb570af16987191b; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-10 accuracy, formula; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-c877ae5cd5; title: MassSpecGym · formula · Table 2; protocol id: paper-protocol-5d42ac5481b1c071e4; dataset id: paper-dataset-3f9a5b3d0316153887; metric: Top-10 MCES; unit: edge-edit distance; direction: lower; result ids: paper-result-b288a91a9b8d3c0ac3; paper-result-0cba6df8abf3e80c3e; paper-result-041cb47a464089a409; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-10 MCES, formula; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table2-c39edfb769; title: MassSpecGym · formula · Table 2; protocol id: paper-protocol-5d42ac5481b1c071e4; dataset id: paper-dataset-3f9a5b3d0316153887; metric: Top-10 Tanimoto; unit: dimensionless; direction: higher; result ids: paper-result-20160e960ae3bce164; paper-result-cc841777ec02cbc26d; paper-result-50fe2681b8dc27e6a7; source ids: part2-massspecgym-arxiv-v1; source locator: Table 2: Top-10 Tanimoto, formula; context: Generate candidate molecules from an input spectrum. Compare within the same main/formula challenge and metric. Main random generation uses precursor mass; the Transformer consumes the spectrum. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-65ea68de77; title: MassSpecGym · main · Table 3; protocol id: paper-protocol-a18b4bc79049513fb7; dataset id: paper-dataset-43f783b5d7a7151aa9; metric: Hit rate @1; unit: percent; direction: higher; result ids: paper-result-9b0cc277a3cbc6c13d; paper-result-acf85ffa57a1906421; paper-result-a0d66c2c9d6bc8466d; paper-result-d73962270e10586574; paper-result-d5320e4453da6b09dd; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: Hit rate @1, main; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-dffdc0b0ef; title: MassSpecGym · main · Table 3; protocol id: paper-protocol-a18b4bc79049513fb7; dataset id: paper-dataset-43f783b5d7a7151aa9; metric: Hit rate @5; unit: percent; direction: higher; result ids: paper-result-35b54face42cd4df98; paper-result-3588b0e1eb7078ee40; paper-result-3bda21118a05d4e278; paper-result-46e2e44bcf768e94b2; paper-result-ed71048c3ebde971d5; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: Hit rate @5, main; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-629ab3b450; title: MassSpecGym · main · Table 3; protocol id: paper-protocol-a18b4bc79049513fb7; dataset id: paper-dataset-43f783b5d7a7151aa9; metric: Hit rate @20; unit: percent; direction: higher; result ids: paper-result-f871d7d35c275e848e; paper-result-e3dc19ea2e9e59e293; paper-result-29a4b30ef091c9e0e8; paper-result-cde595ad3523bf9f27; paper-result-6b9b0e247bfdc9c274; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: Hit rate @20, main; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-b934e3725f; title: MassSpecGym · main · Table 3; protocol id: paper-protocol-a18b4bc79049513fb7; dataset id: paper-dataset-43f783b5d7a7151aa9; metric: MCES @1; unit: edge-edit distance; direction: lower; result ids: paper-result-002aa84a8224be7916; paper-result-3ff182c671377da6da; paper-result-57fd7d0cd8b73c56e2; paper-result-0dcae24f1e2c35b445; paper-result-80aa6ca411586dd8e5; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: MCES @1, main; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-ac4b0b4611; title: MassSpecGym · formula · Table 3; protocol id: paper-protocol-1e7bb17e6ce294d906; dataset id: paper-dataset-e3dee7ae24615eecfb; metric: Hit rate @1; unit: percent; direction: higher; result ids: paper-result-96eb3d9e4116e85dc3; paper-result-19979776fcf90cd085; paper-result-6d5f81ef670db6f81b; paper-result-7e55a965688b10e2cc; paper-result-761f932c6a35c888bb; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: Hit rate @1, formula; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-a71e8b64c9; title: MassSpecGym · formula · Table 3; protocol id: paper-protocol-1e7bb17e6ce294d906; dataset id: paper-dataset-e3dee7ae24615eecfb; metric: Hit rate @5; unit: percent; direction: higher; result ids: paper-result-6bf388327e502fbdd6; paper-result-1b7eee640fec71e99b; paper-result-773438d255878a0817; paper-result-a2021eb25d2e915417; paper-result-7a7bc4ebdca6051342; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: Hit rate @5, formula; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-8dc4756cf6; title: MassSpecGym · formula · Table 3; protocol id: paper-protocol-1e7bb17e6ce294d906; dataset id: paper-dataset-e3dee7ae24615eecfb; metric: Hit rate @20; unit: percent; direction: higher; result ids: paper-result-1816c840f0de8bc10b; paper-result-c905d384ee9e3b5f91; paper-result-e1302d68232a47d02d; paper-result-46121102e555a35255; paper-result-3b6baab5e2664e30e6; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: Hit rate @20, formula; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table3-ebdb34929b; title: MassSpecGym · formula · Table 3; protocol id: paper-protocol-1e7bb17e6ce294d906; dataset id: paper-dataset-e3dee7ae24615eecfb; metric: MCES @1; unit: edge-edit distance; direction: lower; result ids: paper-result-8b2608a402d6ab54e6; paper-result-47e90701446fc3f6b0; paper-result-88df2cfb82ff24f522; paper-result-8d04b85ca056e77ff0; paper-result-e5a6f34b3603af99f9; source ids: part2-massspecgym-arxiv-v1; source locator: Table 3: MCES @1, formula; context: Retrieve the correct molecule from up to 256 candidates using an input spectrum; compare separately within the main challenge and molecular-formula challenge. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-3e7f4289d4; title: MassSpecGym · main · Table 4; protocol id: paper-protocol-8c192f6451f321879d; dataset id: paper-dataset-537c7d5a2018d12af3; metric: Cosine similarity; unit: dimensionless; direction: higher; result ids: paper-result-832c06a3571e8b99ee; paper-result-e6dae78a0e585b9213; paper-result-0310d5160b4254137f; paper-result-6ffb58ff17d94ff067; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Cosine similarity, main; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-c2e7f3b039; title: MassSpecGym · main · Table 4; protocol id: paper-protocol-8c192f6451f321879d; dataset id: paper-dataset-537c7d5a2018d12af3; metric: Jensen-Shannon similarity; unit: dimensionless; direction: higher; result ids: paper-result-bf0caa2f18c9b7b9ba; paper-result-88329ec11a23e189c3; paper-result-cc2224d542c379d64e; paper-result-0c76966663451dc5e3; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Jensen-Shannon similarity, main; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-65ea68de77; title: MassSpecGym · main · Table 4; protocol id: paper-protocol-8c192f6451f321879d; dataset id: paper-dataset-537c7d5a2018d12af3; metric: Hit rate @1; unit: percent; direction: higher; result ids: paper-result-17366d36bcffb780c8; paper-result-665810a5e1f9cada24; paper-result-04b945325754ca6a3c; paper-result-f6b2ddc4c03e718467; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Hit rate @1, main; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-dffdc0b0ef; title: MassSpecGym · main · Table 4; protocol id: paper-protocol-8c192f6451f321879d; dataset id: paper-dataset-537c7d5a2018d12af3; metric: Hit rate @5; unit: percent; direction: higher; result ids: paper-result-2e0c5271342aa65299; paper-result-c82ea8aa8dc5772dfb; paper-result-7d9d8c7b77d1529f6d; paper-result-ad8080ad017d8b7953; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Hit rate @5, main; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-629ab3b450; title: MassSpecGym · main · Table 4; protocol id: paper-protocol-8c192f6451f321879d; dataset id: paper-dataset-537c7d5a2018d12af3; metric: Hit rate @20; unit: percent; direction: higher; result ids: paper-result-33e50c25a480666e26; paper-result-ad0be47f84a1a8fec1; paper-result-0a64456862cd0e89be; paper-result-85739d66e709a022d4; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Hit rate @20, main; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-ac4b0b4611; title: MassSpecGym · formula · Table 4; protocol id: paper-protocol-5d8ed22461594b6dbe; dataset id: paper-dataset-712fb45862f6d8614f; metric: Hit rate @1; unit: percent; direction: higher; result ids: paper-result-0eb930d3d87c1554c8; paper-result-e0a2ae6ef5e9dac842; paper-result-e1dad4454a489eea14; paper-result-89bc901270255db9e9; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Hit rate @1, formula; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-a71e8b64c9; title: MassSpecGym · formula · Table 4; protocol id: paper-protocol-5d8ed22461594b6dbe; dataset id: paper-dataset-712fb45862f6d8614f; metric: Hit rate @5; unit: percent; direction: higher; result ids: paper-result-23f54c2046394a9849; paper-result-2f03b245ed9e4e81e9; paper-result-f1dea3c2c74bc24384; paper-result-f6cbe952ed6e6a1568; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Hit rate @5, formula; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17; id: part2-massspecgym-table4-8dc4756cf6; title: MassSpecGym · formula · Table 4; protocol id: paper-protocol-5d8ed22461594b6dbe; dataset id: paper-dataset-712fb45862f6d8614f; metric: Hit rate @20; unit: percent; direction: higher; result ids: paper-result-a0d4a99203b52c3a11; paper-result-5db072c636cb16b724; paper-result-23d5f2a3887e324717; paper-result-9e337171088dfc4bc7; source ids: part2-massspecgym-arxiv-v1; source locator: Table 4: Hit rate @20, formula; context: Predict a spectrum from a molecule under measurement conditions. Metadata-complete simulation subset; simulation-derived retrieval is distinct from direct molecule retrieval in Table 3. MCES single-linkage molecular clustering at threshold 10; fixed held-out test split; caveats: Source metric equations define hit rates as fractions, while table values use percentage scale; recorded values are not rescaled.; Zero generation accuracy is a reported result, not missingness.; No checkpoint revision inferred from method name.; Bootstrap CIs reflect resampling of this test set; not independently reproduced and not seed standard deviations.; Full table retained, including weaker/random methods and unavailable formula-simulation similarity cells.; review: method: automated_source_review; date: 2026-09-17
benchmark research
review date: 2026-09-17; status: complete_comparison_tables_extracted_pending_publication_review; primary sources: part2-massspecgym-arxiv-v1; inspected locators: Tables2–4; task definitions and supplementary split table; searched queries: MassSpecGym 2410.23326 benchmark results Table 1 Table 2; gaps: Independent batch review before import; preserve existing observation identities.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
historical missing metadata
dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted
metadata review scope
historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
entity classification
review date: 2026-09-17; rationale: The cited profile describes a collection of evaluation tasks or protocols; retain it as the top-level benchmark suite. Its datasets and individual protocols remain separate records.; source ids: src-discovery-pluskal-lab-massspecgym; evidence-benchmark-massspecgym-de-novo-base-py; evidence-benchmark-massspecgym-retrieval-base-py; evidence-benchmark-massspecgym-simulation-base-py; source locator: Pinned README: three challenges; dataset and DataModule; evaluation base classes; pinned de_novo, retrieval and simulation base evaluation classes; ambiguities: None recorded
run guide
record id: discovery-benchmark-massspecgym; summary: Run the official CPU DeepSets retrieval tutorial: train a small spectrum-to-fingerprint model, then evaluate retrieval on the provided test split.; status: source_reviewed_not_executed; prerequisites: Conda and Git; the official README recommends a Python 3.11 environment.; Internet access for MassSpecGym dataset/candidate downloads from Hugging Face. The dataset loader downloads data as needed.; steps: title: Check out the reviewed repository; shell: git clone https://github.com/pluskal-lab/MassSpecGym.git cd MassSpecGym git checkout --detach f259fe3780d5bd227fc6ece36ce6f397c2eef716; explanation: Repository checkout wrapper: the detached revision selects the exact official source inspected for this guide.; source ids: run-doc-massspecgym-readme-md-f259fe37; source locator: Pinned repository revision; README.md; title: Install into the recommended environment; shell: conda create -n massspecgym python==3.11 conda activate massspecgym pip install massspecgym; explanation: The README package-install command is unversioned. Record the installed version; the pinned documentation commit is not an installed-package lock.; source ids: run-doc-massspecgym-readme-md-f259fe37; source locator: README.md lines 38–51; title: Run the documented CPU training and test example; shell: cat > massspecgym_retrieval_example.py <<'PY' import torch import torch.nn as nn import pytorch_lightning as pl from pytorch_lightning import Trainer from massspecgym.data import RetrievalDataset, MassSpecDataModule from massspecgym.data.transforms import SpecTokenizer, MolFingerprinter from massspecgym.models.base import Stage from massspecgym.models.retrieval.base import RetrievalMassSpecGymModel class MyDeepSetsRetrievalModel(RetrievalMassSpecGymModel): def __init__( self, hidden_channels: int = 128, out_channels: int = 4096, # fingerprint size *args, **kwargs ): """Implement your architecture.""" super().__init__(*args, **kwargs) self.phi = nn.Sequential( nn.Linear(2, hidden_channels), nn.ReLU(), nn.Linear(hidden_channels, hidden_channels), nn.ReLU(), ) self.rho = nn.Sequential( nn.Linear(hidden_channels, hidden_channels), nn.ReLU(), nn.Linear(hidden_channels, out_channels), nn.Sigmoid() ) def forward(self, x: torch.Tensor) -> torch.Tensor: """Implement your prediction logic.""" x = self.phi(x) x = x.sum(dim=-2) # sum over peaks x = self.rho(x) return x def step( self, batch: dict, stage: Stage ) -> tuple[torch.Tensor, torch.Tensor]: """Implement your custom logic of using predictions for training and inference.""" # Unpack inputs x = batch["spec"] # input spectra fp_true = batch["mol"] # true fingerprints cands = batch["candidates"] # candidate fingerprints concatenated for a batch batch_ptr = batch["batch_ptr"] # number of candidates per sample in a batch # Predict fingerprint fp_pred = self.forward(x) # Calculate loss loss = nn.functional.mse_loss(fp_true, fp_pred) # Calculate final similarity scores between predicted fingerprints and retrieval candidates fp_pred_repeated = fp_pred.repeat_interleave(batch_ptr, dim=0) scores = nn.functional.cosine_similarity(fp_pred_repeated, cands) return dict(loss=loss, scores=scores) if __name__ == '__main__': # Init hyperparameters n_peaks = 60 fp_size = 4096 batch_size = 32 # Load dataset dataset = RetrievalDataset( spec_transform=SpecTokenizer(n_peaks=n_peaks), mol_transform=MolFingerprinter(fp_size=fp_size), ) # Init data module data_module = MassSpecDataModule( dataset=dataset, batch_size=batch_size, num_workers=4 ) # Init model model = MyDeepSetsRetrievalModel(out_channels=fp_size) # Init trainer trainer = Trainer(accelerator="cpu", devices=1, max_epochs=5) # Train trainer.fit(model, datamodule=data_module) # Test trainer.test(model, datamodule=data_module) PY python massspecgym_retrieval_example.py; explanation: The four official Python blocks are combined in a local script. A standard main guard is added around execution for the documented four data workers; model, CPU trainer, one device, five epochs and batch size 32 are unchanged.; source ids: run-doc-massspecgym-readme-md-f259fe37; source locator: README.md lines 111–218; outputs: Lightning training/validation logs followed by retrieval test metrics from trainer.test on the official data module.; limitations: This is the README tutorial model, not a claim to reproduce a published leaderboard checkpoint.; The example explicitly runs on CPU; runtime, memory demand and storage capacity are not stated.; The PyPI installation, transitive dependencies and downloaded data are not hash-locked. The Python script wrapper has been source-reviewed only, not executed.; source ids: run-doc-massspecgym-readme-md-f259fe37; review: method: official_repository_review; date: 2026-09-17
run documentation
record id: discovery-benchmark-massspecgym; source ids: run-doc-massspecgym-readme-md-f259fe37; status: source_reviewed_not_executed; summary: Run the official CPU DeepSets retrieval tutorial: train a small spectrum-to-fingerprint model, then evaluate retrieval on the provided test split. Commands were source-reviewed only. This is the README tutorial model, not a claim to reproduce a published leaderboard checkpoint.; source locator: Pinned repository revision; README.md; README.md lines 38–51; README.md lines 111–218
run recipes
id: massspecgym-official; protocol id: discovery-benchmark-massspecgym; version: f259fe3780d5bd227fc6ece36ce6f397c2eef716; title: Load MassSpecGym and evaluate a model; purpose: generate_and_evaluate; summary: Install the package and load the benchmark dataset that the scores on this page are measured over.; inputs: A model for one of the benchmark's spectrum tasks.; outputs: The benchmark dataset and the scores its evaluation produces.; requirements: data: Downloaded by the package on first use.; weights: None required.; licence: Project licence: MIT. Upstream data licences are separate and unreported here.; software: Python with the massspecgym package from PyPI.; hardware: Not stated in the cited section. Several of these steps expect a GPU.; instructions: runtime: command_line; title: Install; code: pip install massspecgym; status: source_reviewed_not_executed; source ids: project-recipe-massspecgym-f259fe37; source locator: README.md at f259fe37, Installation, lines 43-43; runtime: python; title: Load the dataset; code: from massspecgym.utils import load_massspecgym df = load_massspecgym(); status: source_reviewed_not_executed; source ids: project-recipe-massspecgym-f259fe37; source locator: README.md at f259fe37, Getting started with MassSpecGym, lines 76-77; limitations: Quoted from the project's README and not executed by rewire, so the commands are evidence of what the project documents rather than a verified run.; The project may have changed since the pinned commit.; The tasks use different metrics and are scored separately.; source ids: project-recipe-massspecgym-f259fe37; source locator: README.md at f259fe37
Related records

Suggest a correction