PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusion

Abstract: We present PepTune, a multi-objective discrete diffusion model for simultaneous generation and optimization of therapeutic peptide SMILES. Built on the Masked Discrete LLM (MDLM) framework, PepTune ensures valid peptide structures with a novel bond-dependent masking schedule and invalid loss function. To guide the diffusion process, we introduce Monte Carlo Tree Guidance (MCTG), an inference-time multi-objective guidance algorithm that balances exploration and exploitation to iteratively refine Pareto-optimal sequences. MCTG integrates classifier-based rewards with search-tree expansion, overcoming gradient estimation challenges and data sparsity. Using PepTune, we generate diverse, chemically-modified peptides simultaneously optimized for multiple therapeutic properties, including target binding affinity, membrane permeability, solubility, hemolysis, and non-fouling for various disease-relevant targets. In total, our results demonstrate that MCTG for masked discrete diffusion is a powerful and modular approach for multi-objective sequence design in discrete state spaces.

• The paper introduces PepTune, a discrete diffusion model guided by MCTS for multi-objective therapeutic peptide generation.
• PepTune utilizes techniques like state-dependent masking and an invalidity penalty to ensure generated peptide sequences are chemically valid.
• PepTune generates diverse peptides optimized for multiple properties, accelerating therapeutic design and suggesting applications in other biomolecular fields.

Overview of PepTune: Multi-Objective Optimization for Therapeutic Peptide Design

The paper introduces PepTune, a discrete diffusion model designed to generate and optimize therapeutic peptides conditioned on multiple complex objectives. These peptides, expressed as SMILES strings, are pivotal for various therapeutic applications, from diabetes to cancer treatments. However, developing them poses significant challenges due to the need to satisfy multiple conflicting properties such as binding affinity, solubility, and membrane permeability.

PepTune utilizes the Masked Discrete LLM (MDLM) framework, underpinned by a sophisticated Monte Carlo Tree Search (MCTS)-based strategy. This architecture facilitates the exploration of peptide sequences ensuring valid chemical structures with state-dependent masking, while balancing conflicting objectives using a Pareto-optimal approach. The study introduces several methodological innovations, including a penalty-based objective to ensure the generation of syntactically and chemically sound peptide sequences.

Methodological Contributions

1. State-Dependent Masking: The paper introduces a state-dependent masking schedule. This schedule specifically controls the diffusion process, ensuring that peptide bond tokens hold higher priority during sequence generation. This approach enhances the model's ability to generate chemically valid peptide structures.
2. Monte Carlo Tree Search (MCTS): PepTune integrates an MCTS-based guidance mechanism to steer the generative model towards Pareto-optimal sequences. This sophisticated exploration-exploitation strategy is key to effectively balancing multiple objectives such as binding affinity and membrane permeability.
3. Invalidity Penalty: A globally integrated sequence invalidity penalty is introduced to penalize predicted token probabilities resulting in invalid SMILES strings. This objective aids in maintaining the structural and chemical integrity of the generated peptides.
4. Property Prediction Toolkit: The study also contributes a robust toolkit for predicting properties of peptide SMILES. This toolkit encompasses both regression and classification models to evaluate key therapeutic properties, which are used to inform the MCTS-based guidance.

Results and Implications

PepTune demonstrates significant efficacy in generating a diverse set of peptides adapted for multiple therapeutic properties across various disease-relevant proteins. The model achieves this by exploring the space of peptide SMILES strings, ensuring sequences are chemically valid and tailored to satisfy multiple therapeutic properties, thus proving to be highly modular and adaptable to complex peptide design tasks.

The introduction of PepTune has several implications:

• Theoretical Implications: By demonstrating that discrete diffusion models can effectively handle multi-objective optimization in sequence design, the paper opens new avenues for research in generative modeling across other fields.
• Practical Implications: The multi-objective capacity of PepTune can accelerate therapeutic peptide design, reducing time and resource investments significantly. This capability is crucial in fields such as personalized medicine and targeted drug delivery where specific peptide interactions are desired.
• Future Directions: The study suggests potential expansions of PepTune's methodology to other bio-molecular design challenges, including DNA and protein sequences, and beyond into areas such as materials science where the design of complex, multifunctional structures is needed.

In conclusion, the paper presents PepTune as a significant stride in therapeutic peptide design, effectively addressing the inherent challenges of multi-objective optimization. This research represents a sophisticated blend of model precision and practical relevance, offering a promising tool for future developments in bioengineering and pharmaceutical sciences.