Papers
Topics
Authors
Recent
Search
2000 character limit reached

TaxDiff: Taxonomic-Guided Diffusion Model for Protein Sequence Generation

Published 27 Feb 2024 in cs.LG, cs.AI, and q-bio.BM | (2402.17156v1)

Abstract: Designing protein sequences with specific biological functions and structural stability is crucial in biology and chemistry. Generative models already demonstrated their capabilities for reliable protein design. However, previous models are limited to the unconditional generation of protein sequences and lack the controllable generation ability that is vital to biological tasks. In this work, we propose TaxDiff, a taxonomic-guided diffusion model for controllable protein sequence generation that combines biological species information with the generative capabilities of diffusion models to generate structurally stable proteins within the sequence space. Specifically, taxonomic control information is inserted into each layer of the transformer block to achieve fine-grained control. The combination of global and local attention ensures the sequence consistency and structural foldability of taxonomic-specific proteins. Extensive experiments demonstrate that TaxDiff can consistently achieve better performance on multiple protein sequence generation benchmarks in both taxonomic-guided controllable generation and unconditional generation. Remarkably, the sequences generated by TaxDiff even surpass those produced by direct-structure-generation models in terms of confidence based on predicted structures and require only a quarter of the time of models based on the diffusion model. The code for generating proteins and training new versions of TaxDiff is available at:https://github.com/Linzy19/TaxDiff.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Uniprot: the universal protein knowledgebase in 2023. Nucleic Acids Research, 51(D1):D523–D531, 2023.
  2. Protein generation with evolutionary diffusion: sequence is all you need. bioRxiv, pp.  2023–09, 2023.
  3. Generative modeling for protein structures. Advances in neural information processing systems, 31, 2018.
  4. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981–17993, 2021.
  5. Accurate prediction of protein structures and interactions using a three-track neural network. Science, 373(6557):871–876, 2021.
  6. The protein data bank. Acta Crystallographica Section D: Biological Crystallography, 58(6):899–907, 2002.
  7. Low-n protein engineering with data-efficient deep learning. Nature methods, 18(4):389–396, 2021.
  8. Fold2seq: A joint sequence (1d)-fold (3d) embedding-based generative model for protein design. In International Conference on Machine Learning, pp.  1261–1271. PMLR, 2021.
  9. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  10. Ig-vae: Generative modeling of protein structure by direct 3d coordinate generation. PLoS computational biology, 18(6):e1010271, 2022.
  11. Protgpt2 is a deep unsupervised language model for protein design. Nature communications, 13(1):4348, 2022.
  12. Improving catalytic function by prosar-driven enzyme evolution. Nature biotechnology, 25(3):338–344, 2007.
  13. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677, 2017.
  14. Generating functional protein variants with variational autoencoders. PLoS computational biology, 17(2):e1008736, 2021.
  15. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
  16. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  17. The coming of age of de novo protein design. Nature, 537(7620):320–327, 2016.
  18. Illuminating protein space with a programmable generative model. Nature, pp.  1–9, 2023.
  19. Diffusionret: Generative text-video retrieval with diffusion model. arXiv preprint arXiv:2303.09867, 2023.
  20. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589, 2021.
  21. Generating novel, designable, and diverse protein structures by equivariantly diffusing oriented residue clouds. arXiv preprint arXiv:2301.12485, 2023.
  22. Deep generative models create new and diverse protein structures. In Machine Learning for Structural Biology Workshop, NeurIPS, 2021.
  23. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022:500902, 2022.
  24. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, 2023.
  25. Generating 3d molecules for target protein binding. arXiv preprint arXiv:2204.09410, 2022.
  26. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  27. Luby, J. J. Taxonomic classification and brief history. In Apples: botany, production and uses, pp.  1–14. Cabi Publishing Wallingford UK, 2003.
  28. Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures. Advances in Neural Information Processing Systems, 35:9754–9767, 2022.
  29. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021.
  30. Progen2: exploring the boundaries of protein language models. Cell Systems, 14(11):968–978, 2023.
  31. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  4195–4205, 2023.
  32. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  33. Generating immune-aware sars-cov-2 spike proteins for universal vaccine design. In Workshop on Healthcare AI and COVID-19, pp.  100–116. PMLR, 2022.
  34. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023.
  35. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  36. Msa transformer. In International Conference on Machine Learning, pp.  8844–8856. PMLR, 2021.
  37. Expanding functional protein sequence spaces using generative adversarial networks. Nature Machine Intelligence, 3(4):324–333, 2021.
  38. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  10684–10695, 2022.
  39. Exploring protein fitness landscapes by directed evolution. Nature reviews Molecular cell biology, 10(12):866–876, 2009.
  40. Protein design and variant prediction using autoregressive generative models. Nature communications, 12(1):2403, 2021.
  41. Freeu: Free lunch in diffusion u-net. arXiv preprint arXiv:2309.11497, 2023.
  42. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp.  2256–2265. PMLR, 2015.
  43. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  44. Importance weighted expectation-maximization for protein sequence design. arXiv preprint arXiv:2305.00386, 2023.
  45. De novo creation of fluorescent molecules via adversarial generative modeling. RSC advances, 13(2):1031–1040, 2023.
  46. Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. arXiv preprint arXiv:2206.04119, 2022.
  47. Fast and accurate protein structure search with foldseek. Nature Biotechnology, pp.  1–4, 2023.
  48. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic acids research, 50(D1):D439–D444, 2022.
  49. De novo design of protein structure and function with rfdiffusion. Nature, 620(7976):1089–1100, 2023.
  50. Database resources of the national center for biotechnology information. Nucleic acids research, 35(suppl_1):D5–D12, 2007.
  51. Protein structure generation via folding diffusion. arXiv preprint arXiv:2209.15611, 2022a.
  52. High-resolution de novo structure prediction from primary sequence. bioRxiv, 2022b. doi: 10.1101/2022.07.21.500999. URL https://www.biorxiv.org/content/early/2022/07/22/2022.07.21.500999.
  53. Signal peptides generated by attention-based neural networks. ACS Synthetic Biology, 9(8):2154–2161, 2020.
  54. Convolutions are competitive with transformers for protein sequence pretraining. bioRxiv, pp.  2022–05, 2022.
  55. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4):1–39, 2023.
  56. Se (3) diffusion model with application to protein backbone generation. arXiv preprint arXiv:2302.02277, 2023.
  57. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  3836–3847, 2023.
  58. Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 57(4):702–710, 2004.
  59. Structure-informed language models are protein designers. bioRxiv, pp.  2023–02, 2023.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free