Papers
Topics
Authors
Recent
Search
2000 character limit reached

De novo Drug Design using Reinforcement Learning with Multiple GPT Agents

Published 21 Dec 2023 in q-bio.BM, cs.CE, and cs.LG | (2401.06155v1)

Abstract: De novo drug design is a pivotal issue in pharmacology and a new area of focus in AI for science research. A central challenge in this field is to generate molecules with specific properties while also producing a wide range of diverse candidates. Although advanced technologies such as transformer models and reinforcement learning have been applied in drug design, their potential has not been fully realized. Therefore, we propose MolRL-MGPT, a reinforcement learning algorithm with multiple GPT agents for drug molecular generation. To promote molecular diversity, we encourage the agents to collaborate in searching for desirable molecules in diverse directions. Our algorithm has shown promising results on the GuacaMol benchmark and exhibits efficacy in designing inhibitors against SARS-CoV-2 protein targets. The codes are available at: https://github.com/HXYfighter/MolRL-MGPT.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Guiding deep molecular optimization with genetic exploration. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 12008–12021. Curran Associates, Inc., 2020.
  2. Fast, accurate, and reliable molecular docking with quickvina 2. Bioinformatics, 31(13):2214–2216, 2015.
  3. Randomized smiles strings improve the quality of molecular generative models. Journal of cheminformatics, 11(1):1–13, 2019.
  4. Molgpt: Molecular generation using a transformer-decoder model. Journal of Chemical Information and Modeling, 62(9):2064–2076, 2022.
  5. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021.
  6. Mostapha Benhenda. Can ai reproduce observed chemical diversity? bioRxiv, page 292177, 2018.
  7. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90–98, 2012.
  8. Reinvent 2.0: An ai tool for de novo drug design. Journal of chemical information and modeling, 2020.
  9. Memory-assisted reinforcement learning for diverse molecular de novo design. Journal of chemical information and modeling, 2020.
  10. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3):1096–1108, 2019.
  11. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  12. Molecular diversity in drug design. Springer, 1999.
  13. Artificial intelligence in drug discovery: applications and techniques. Briefings in Bioinformatics, 23(1):bbab430, 2022.
  14. A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications. Artificial Intelligence Review, 54:3215–3238, 2021.
  15. Molgensurvey: A systematic survey in machine learning models for molecule design. arXiv preprint arXiv:2203.14500, 2022.
  16. Limo: Latent inceptionism for targeted molecule generation. In International Conference on Machine Learning. PMLR, 2022.
  17. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1(1):1–11, 2009.
  18. Reinforced genetic algorithm for structure-based drug design. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems NeurIPS, 2022.
  19. Moler: incorporate molecule-level reward to enhance deep generative model for molecule optimization. IEEE transactions on knowledge and data engineering, 34(11):5459–5471, 2021.
  20. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. Advances in neural information processing systems, 32, 2019.
  21. De novo molecular generation via connection-aware motif mining. In International Conference on Learning Representations, 2023.
  22. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268–276, 2018.
  23. Objective-reinforced generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv:1705.10843, 2017.
  24. Transformer-based molecular optimization beyond matched molecular pairs. Journal of cheminformatics, 14(1):18, 2022.
  25. Structure of replicating sars-cov-2 polymerase. Nature, 584(7819):154–156, 2020.
  26. Chemformer: a pre-trained transformer for computational chemistry. Machine Learning: Science and Technology, 3(1):015022, 2022.
  27. Is gpt-3 all you need for machine learning for chemistry? In AI for Accelerated Materials Design NeurIPS 2022 Workshop, 2022.
  28. Jan H Jensen. A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567–3572, 2019.
  29. Multi-objective molecule generation using interpretable substructures. In International Conference on Machine Learning, page 4849–4859. PMLR, 2020.
  30. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pages 2323–2332. PMLR, 2018.
  31. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10(1):1–24, 2018.
  32. Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 8(3):293–321, 1992.
  33. An autoregressive flow model for 3d molecular geometry generation from scratch. In International Conference on Learning Representations (ICLR), 2022.
  34. Maven: Multi-agent variational exploration. Advances in Neural Information Processing Systems, 32, 2019.
  35. Learning to extend molecular scaffolds with structural motifs. In International Conference on Learning Representations (ICLR), 2022.
  36. Chembl: towards direct deposition of bioassay data. Nucleic acids research, 47(D1):D930–D940, 2019.
  37. Boss: Bayesian optimization over string spaces. Advances in neural information processing systems, 33:15476–15486, 2020.
  38. Advances in de novo drug design: From conventional to machine learning methods. International journal of molecular sciences, 22(4):1676, 2021.
  39. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics, 9, 2017.
  40. Structure of papain-like protease from sars-cov-2 and its complexes with non-covalent inhibitors. Nature communications, 12(1):743, 2021.
  41. Diversity oriented deep reinforcement learning for targeted molecule generation. Journal of cheminformatics, 13(1):1–17, 2021.
  42. Estimation of the size of drug-like chemical space based on gdb-17 data. Journal of computer-aided molecular design, 27(8):675–679, 2013.
  43. Molecular sets (moses): a benchmarking platform for molecular generation models. Frontiers in pharmacology, 11:565644, 2020.
  44. Alphadrug: protein target specific de novo molecular generation. PNAS Nexus, 1(4):pgac227, 2022.
  45. Improving language understanding by generative pre-training. 2018.
  46. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  47. Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 50(5):742–754, 2010.
  48. Sars-cov2 billion-compound docking. Scientific Data, 10(1):173, 2023.
  49. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1):120–131, 2018.
  50. Planning chemical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):604–610, 2018.
  51. Reinforcement learning for molecular design guided by quantum mechanics. In International Conference on Machine Learning, pages 8959–8969. PMLR, 2020.
  52. Symmetry-aware actor-critic for 3d molecular design. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  53. Zinc 15 – ligand discovery for everyone. Journal of chemical information and modeling, 15(11):2324–2337, 2015.
  54. Excape-db: an integrated large scale dataset facilitating big data analysis in chemogenomics. Journal of cheminformatics, 9(1):1–9, 2017.
  55. T. T. Tanimoto. An elementary mathematical theory of classification and prediction. IBM Internal Report, 1958.
  56. Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):455–461, 2010.
  57. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  58. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023.
  59. Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning. Nature Machine Intelligence, 3(10):914–922, 2021.
  60. Relation: A deep generative model for structure-based de novo drug design. Journal of Medicinal Chemistry, 65(13):9478–9492, 2022.
  61. Tailoring molecules for protein pockets: a transformer-based generative solution for structured-based drug design. arXiv preprint arXiv:2209.06158, 2022.
  62. Mars: Markov molecular sampling for multi-objective drug discovery. In International Conference on Learning Representations, 2021.
  63. Hit and lead discovery with explorative rl and fragment-based molecule generation. Advances in Neural Information Processing Systems, 34:7924–7936, 2021.
  64. Population-based de novo molecule generation, using grammatical evolution. Chemistry Letters, 47(11):1431–1434, 2018.
  65. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018.
  66. Advancing computer-aided drug discovery (cadd) by big data and data-driven machine learning modeling. Drug Discovery Today, 25(9):1624–1638, 2020.
  67. Accelerated rational protac design via deep learning and molecular simulations. Nature Machine Intelligence, 4(9):739–748, 2022.
  68. Optimization of molecules via deep reinforcement learning. Scientific Reports, 9, 2019.
Citations (11)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 2 tweets with 1 like about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free