MassSpecGym: A benchmark for the discovery and identification of molecules

Abstract: The discovery and identification of molecules in biological and environmental samples is crucial for advancing biomedical and chemical sciences. Tandem mass spectrometry (MS/MS) is the leading technique for high-throughput elucidation of molecular structures. However, decoding a molecular structure from its mass spectrum is exceptionally challenging, even when performed by human experts. As a result, the vast majority of acquired MS/MS spectra remain uninterpreted, thereby limiting our understanding of the underlying (bio)chemical processes. Despite decades of progress in machine learning applications for predicting molecular structures from MS/MS spectra, the development of new methods is severely hindered by the lack of standard datasets and evaluation protocols. To address this problem, we propose MassSpecGym -- the first comprehensive benchmark for the discovery and identification of molecules from MS/MS data. Our benchmark comprises the largest publicly available collection of high-quality labeled MS/MS spectra and defines three MS/MS annotation challenges: de novo molecular structure generation, molecule retrieval, and spectrum simulation. It includes new evaluation metrics and a generalization-demanding data split, therefore standardizing the MS/MS annotation tasks and rendering the problem accessible to the broad machine learning community. MassSpecGym is publicly available at https://github.com/pluskal-lab/MassSpecGym.

• The paper presents the first comprehensive MS/MS benchmark, tackling challenges in molecule discovery and structure identification.
• It leverages a curated dataset of 231,000 spectra covering 29,000 molecules, with standardized tasks for de novo structure generation, molecular retrieval, and spectrum simulation.
• The benchmark employs a novel molecular edit distance splitting method to ensure robust generalization and foster community-driven improvements.

MassSpecGym: A Novel Benchmark for Molecule Discovery and Identification Using MS/MS Data

The paper "MassSpecGym: A benchmark for the discovery and identification of molecules" presents a significant contribution to the field of computational metabolomics by addressing pressing issues in mass spectrometry-based molecule identification. Despite the advancements in tandem mass spectrometry (MS/MS) for structural elucidation of compounds within biological and environmental samples, the process remains complex, with a large portion of spectra often left uninterpreted. This lack of comprehensive molecular annotations impedes scientific progress across disciplines such as drug discovery, clinical diagnostics, and environmental analysis.

MassSpecGym proposes to fill a critical gap in the current landscape of mass spectrometry by introducing the first comprehensive benchmark tailored towards MS/MS data annotation. The authors identify several key challenges hampering the advancement of machine learning applications in this domain: heterogeneity and scarcity of high-quality data, inconsistent data preprocessing methods, and the absence of standardized datasets and evaluation protocols. To address these, MassSpecGym leverages the largest publicly accessible collection of high-quality labeled MS/MS spectra and sets standardized tasks with rigorous evaluation metrics.

Key features of the MassSpecGym benchmark include:

• Three Annotation Challenges: The benchmark comprises three core tasks: de novo molecular structure generation from MS/MS spectra, molecular retrieval from chemical databases based on spectra, and spectrum simulation from molecular structures. These tasks are designed to abstract the process of molecule discovery into tangible problems suitable for machine learning models.
• Robust Dataset: MassSpecGym provides a dataset of 231,000 high-quality labeled MS/MS spectra, covering 29,000 unique molecular structures. This dataset is distinguished by its careful curation and standardization across metadata such as ionization adducts and instrument types.
• Novel Data Splitting Methodology: To ensure robust model generalization, the dataset is split based on molecular edit distances using the maximum common edge subgraph (MCES) approach, preventing data leakage. This methodology ensures that model evaluation reflects true capability in generalizing to novel molecular structures.
• Open Access and Community Engagement: The benchmark is openly accessible via platforms like PyTorch Lightning and Hugging Face, providing an interface that requires minimal domain expertise to engage with, thereby widening participation from the machine learning community.

The study results indicate a wide margin for improvements in current methodologies. Baseline models, including state-of-the-art approaches, show varied performance across the tasks, yet highlight significant room for enhancement, particularly in the challenging de novo molecular generation task where achieving exact molecular predictions is notably difficult.

In practical terms, MassSpecGym has the potential to catalyze the development of more accurate and scalable algorithms for MS/MS spectrum annotation. By allowing a broader machine learning audience to engage with the challenges presented, this benchmark can drive methodological innovation and contribute to uncovering the "dark matter" of untapped molecular space.

Looking forward, the paper discusses plans to enhance MassSpecGym by incorporating additional spectral data types and expanding the scope of challenges. Developing more sophisticated models using advanced machine learning paradigms such as graph neural networks and diffusion models for molecule generation constitutes a promising direction for future work. MassSpecGym not only sets a foundation for advancing computational mass spectrometry but also facilitates interdisciplinary collaborations that span analytical chemistry and artificial intelligence, promising significant strides in metabolomics and related fields.