Efficient and Scalable Graph Generation through Iterative Local Expansion
Abstract: In the realm of generative models for graphs, extensive research has been conducted. However, most existing methods struggle with large graphs due to the complexity of representing the entire joint distribution across all node pairs and capturing both global and local graph structures simultaneously. To overcome these issues, we introduce a method that generates a graph by progressively expanding a single node to a target graph. In each step, nodes and edges are added in a localized manner through denoising diffusion, building first the global structure, and then refining the local details. The local generation avoids modeling the entire joint distribution over all node pairs, achieving substantial computational savings with subquadratic runtime relative to node count while maintaining high expressivity through multiscale generation. Our experiments show that our model achieves state-of-the-art performance on well-established benchmark datasets while successfully scaling to graphs with at least 5000 nodes. Our method is also the first to successfully extrapolate to graphs outside of the training distribution, showcasing a much better generalization capability over existing methods.
- Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325, 2023.
- Statistical mechanics of complex networks. Rev. Mod. Phys., 74:47ā97, 01 2002.
- BrianĀ D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313ā326, 1982. ISSN 0304-4149.
- Diffusion visual counterfactual explanations. Advances in Neural Information Processing Systems, 35:364ā377, 2022.
- Structured denoising diffusion models in discrete state-spaces. CoRR, 2107.03006, 2021.
- Layer normalization, 2016.
- Discovering graph generation algorithms. arXiv preprint arXiv:2304.12895, 2023.
- Learning to execute programs with instruction pointer attention graph neural networks. ArXiv, 2010.12621, 2020.
- Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10):P10008, oct 2008.
- Audiolm: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
- Molgan: An implicit generative model for small molecular graphs, 2022.
- Analog bits: Generating discrete data using diffusion models with self-conditioning, 2023a.
- Efficient and degree-guided graph generation via discrete diffusion modeling. ArXiv, 2305.04111, 2023b.
- E.Ā Cuthill and J.Ā McKee. Reducing the bandwidth of sparse symmetric matrices. In Proceedings of the 1969 24th National Conference, ACM ā69, pp.Ā 157ā172, New York, NY, USA, 1969. Association for Computing Machinery. ISBN 9781450374934.
- Scalable deep generative modeling for sparse graphs. In HalĀ DaumĆ© III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.Ā 2302ā2312. PMLR, 13ā18 Jul 2020.
- Size matters: Large graph generation with higgs, 2023.
- Improving graph generation by restricting graph bandwidth, 2023.
- Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology, 330:771ā83, 08 2003.
- Unicats: A unified context-aware text-to-speech framework with contextual vq-diffusion and vocoding. arXiv preprint arXiv:2306.07547, 2023.
- A generalization of transformer networks to graphs, 2021.
- On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci, 5(1):17ā60, 1960.
- Graph neural networks for social recommendation. The World Wide Web Conference, 2019.
- Fast graph representation learning with pytorch geometric, 2019.
- Automating rigid origami design. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pp.Ā 5815ā5823, 8 2023. AI and Arts.
- Graphgen: A scalable approach to domain-agnostic labeled graph generation. In Proceedings of The Web Conference 2020. ACM, 2020.
- Graphite: Iterative generative modeling of graphs, 2019.
- Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.Ā 10696ā10706, 2022.
- An unpooling layer for graph generation, 2023.
- Diffusion models for graphs benefit from discrete state spaces, 2023.
- Exploring network structure, dynamics, and function using networkx. In GaĆ«l Varoquaux, Travis Vaught, and Jarrod Millman (eds.), Proceedings of the 7th Python in Science Conference, pp.Ā 11 ā 15, Pasadena, CA USA, 2008.
- A unifying framework for spectrum-preserving graph sparsification and coarsening. ArXiv, 1902.09702, 2019.
- Denoising diffusion probabilistic models, 2020.
- Stochastic blockmodels: First steps. Social Networks, 5(2):109ā137, 1983. ISSN 0378-8733.
- Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454ā12465, 2021.
- Strategies for pre-training graph neural networks, 2020.
- Illuminating protein space with a programmable generative model. bioRxiv, 2022.
- Junction tree variational autoencoder for molecular graph generation. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volumeĀ 80 of Proceedings of Machine Learning Research, pp.Ā 2323ā2332. PMLR, 10ā15 Jul 2018.
- Hierarchical generation of molecular graphs using structural motifs. In International Conference on Machine Learning, 2020a.
- Graph coarsening with preserved spectral properties. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp.Ā 4452ā4462. PMLR, 26ā28 Aug 2020b.
- Graph generation with destination-predicting diffusion mixture. 2023.
- Mahdi Karami. Higen: Hierarchical graph generative networks, 2023.
- Elucidating the design space of diffusion-based generative models, 2022.
- Adam: A method for stochastic optimization, 2017.
- Autoregressive diffusion model for graph generation, 2023.
- A unified framework for optimization-based graph coarsening. J. Mach. Learn. Res., 24:118:1ā118:50, 2022.
- Featured graph coarsening with similarity guarantees. In International Conference on Machine Learning, 2023.
- Molgrow: A graph normalizing flow for hierarchical molecular generation, 2021.
- Efficient graph generation with graph recurrent attention networks, 2020.
- Sign and basis invariant networks for spectral graph representation learning, 2022.
- Sagess: Sampling graph denoising diffusion model for scalable graph generation, 2023.
- Graph normalizing flows, 2019.
- Andreas Loukas. Graph reduction with spectral and cut guarantees. J. Mach. Learn. Res., 20:116:1ā116:42, 2018.
- Spectrally approximating large graphs with smaller graphs. ArXiv, 1802.07510, 2018.
- Provably powerful graph networks, 2020.
- Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators, 2022.
- Abdiffuser: Full-atom generation of in-vitro functioning antibodies. ArXiv, 2308.05027, 2023.
- Graph kernels for object category prediction in task-dependent robot grasping. In Mining and Learning with Graphs, 2013.
- Graph-based pattern-oriented, context-sensitive source code completion. 2012 34th International Conference on Software Engineering (ICSE), pp.Ā 69ā79, 2012.
- Permutation invariant graph generation via score-based generative modeling, 2020.
- Td-gen: Graph generation with tree decomposition, 2022.
- Graphvae: Towards generation of small graphs using variational autoencoders, 2018.
- Deep unsupervised learning using nonequilibrium thermodynamics, 2015.
- Score-based generative modeling through stochastic differential equations, 2021.
- Improved vector quantized diffusion models. arXiv preprint arXiv:2205.16007, 2022.
- Attention is all you need, 2023.
- Digress: Discrete denoising diffusion for graph generation, 2023a.
- Midi: Mixed graph and 3d denoising diffusion for molecule generation. In ECML/PKDD, 2023b.
- Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661ā1674, 2011.
- NisheethĀ K. Vishnoi. Lx = b. Foundations and TrendsĀ® in Theoretical Computer Science, 8(1ā2):1ā141, 2013. ISSN 1551-305X.
- How powerful are graph neural networks?, 2019.
- Swingnn: Rethinking permutation invariance in diffusion models for graph generation, 2023.
- Graphrnn: Generating realistic graphs with deep auto-regressive models, 2018.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.