Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Theory of Non-Acyclic Generative Flow Networks

Published 23 Dec 2023 in cs.LG, cs.AI, cs.NA, math.NA, and math.PR | (2312.15246v1)

Abstract: GFlowNets is a novel flow-based method for learning a stochastic policy to generate objects via a sequence of actions and with probability proportional to a given positive reward. We contribute to relaxing hypotheses limiting the application range of GFlowNets, in particular: acyclicity (or lack thereof). To this end, we extend the theory of GFlowNets on measurable spaces which includes continuous state spaces without cycle restrictions, and provide a generalization of cycles in this generalized context. We show that losses used so far push flows to get stuck into cycles and we define a family of losses solving this issue. Experiments on graphs and continuous tasks validate those principles.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org.
  2. Gradient Flows: In Metric Spaces and in the Space of Probability Measures. Lectures in Mathematics. ETH Zürich. Birkhäuser Basel. ISBN 9783764373092.
  3. Smart darting monte carlo. The Journal of Chemical Physics, 114(16): 6994–7000.
  4. A tutorial on adaptive MCMC. Statistics and computing, 18(4): 343–373.
  5. A distributional perspective on reinforcement learning. In International Conference on Machine Learning, 449–458. PMLR.
  6. Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation. arXiv:2106.04399.
  7. GFlowNet Foundations. arXiv:2111.09266.
  8. Bogachev, V. 2007. Measure Theory Volume, volume 1 and 2.
  9. OpenAI Gym. arXiv:1606.01540.
  10. Handbook of Markov chain Monte Carlo. CRC press.
  11. Generative Flow Networks: a Markov Chain Perspective. arXiv preprint arXiv:2307.01422.
  12. Markov chains. Springer.
  13. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699.
  14. Neural Topological Ordering for Computation Graphs. arXiv preprint arXiv:2207.05899.
  15. Biological sequence design with gflownets. In International Conference on Machine Learning, 9786–9801. PMLR.
  16. Multi-Objective GFlowNets. arXiv preprint arXiv:2210.12765.
  17. Kalpazidou, S. L. 2007. Cycle representations of Markov processes, volume 28. Springer Science & Business Media.
  18. Kifer, Y. 2012. Ergodic theory of random transformations, volume 10. Springer Science & Business Media.
  19. A theory of continuous generative flow networks. arXiv preprint arXiv:2301.12594.
  20. DAG Matters! GFlowNets Enhanced Explainer for Graph Neural Networks. In The Eleventh International Conference on Learning Representations.
  21. Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphs. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23. International Joint Conferences on Artificial Intelligence Organization.
  22. GFlowNets with Human Feedback. In The First Tiny Papers Track at ICLR 2023, Tiny Papers @ ICLR 2023, Kigali, Rwanda, May 5, 2023. OpenReview.net.
  23. CFlowNets: Continuous Control with Generative Flow Networks. In The Eleventh International Conference on Learning Representations.
  24. Learning GFlowNets from partial episodes for improved convergence and stability. arXiv preprint arXiv:2209.12782.
  25. Trajectory Balance: Improved Credit Assignment in GFlowNets. arXiv preprint arXiv:2201.13259.
  26. Stochastic Generative Flow Networks. arXiv preprint arXiv:2302.09465.
  27. Information Theory From Coding to Learning. Cambridge university press.
  28. A framework for adaptive MCMC targeting multimodal distributions. The Annals of Statistics, 48(5): 2930 – 2952.
  29. Local-Global MCMC kernels: the best of both worlds. arXiv preprint arXiv:2111.02702.
  30. Reinforcement learning: An introduction. MIT press.
  31. Gymnasium.
  32. Metropolis light transport. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, 65–76.
  33. Regularized Offline GFlowNets. In The First Tiny Papers Track at ICLR 2023, Tiny Papers @ ICLR 2023, Kigali, Rwanda, May 5, 2023. OpenReview.net.
  34. Unifying generative models with gflownets. arXiv preprint arXiv:2209.02606.
  35. Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets. arXiv preprint arXiv:2305.17010.
  36. Robust scheduling with GFlowNets. arXiv preprint arXiv:2302.05446.
  37. Generalized Universal Domain Adaptation with Generative Flow Networks. In Proceedings of the 31st ACM International Conference on Multimedia.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up