Papers
Topics
Authors
Recent
Search
2000 character limit reached

Nonlinear Sheaf Diffusion in Graph Neural Networks

Published 1 Mar 2024 in cs.LG | (2403.00337v1)

Abstract: This work focuses on exploring the potential benefits of introducing a nonlinear Laplacian in Sheaf Neural Networks for graph-related tasks. The primary aim is to understand the impact of such nonlinearity on diffusion dynamics, signal propagation, and performance of neural network architectures in discrete-time settings. The study primarily emphasizes experimental analysis, using real-world and synthetic datasets to validate the practical effectiveness of different versions of the model. This approach shifts the focus from an initial theoretical exploration to demonstrating the practical utility of the proposed model.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. The surprising power of graph neural networks with random node initialization. arXiv preprint arXiv:2010.01179, 2020.
  2. Sheaf attention networks. In NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations, 2022.
  3. The physics of higher-order interactions in complex systems. Nature Physics, 17(10):1093–1098, 2021.
  4. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in neural information processing systems, 14, 2001.
  5. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in gnns. Advances in Neural Information Processing Systems, 35:18527–18541, 2022.
  6. Netgan: Generating graphs via random walks. In International conference on machine learning, pages 610–619. PMLR, 2018.
  7. Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks. In Computer graphics forum, volume 34, pages 13–23. Wiley Online Library, 2015.
  8. Residual gated graph convnets. arXiv preprint arXiv:1711.07553, 2017.
  9. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021.
  10. Jason Brownlee. A gentle introduction to the rectified linear unit (relu). Machine learning mastery, 6, 2019.
  11. Low-dimensional hyperbolic knowledge graph embeddings. arXiv preprint arXiv:2005.00545, 2020.
  12. Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247, 2018.
  13. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 257–266, 2019.
  14. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
  15. Justin Michael Curry. Sheaves, cosheaves and applications. University of Pennsylvania, 2014.
  16. Mixing beliefs among interacting agents. Advances in Complex Systems, 3(01n04):87–98, 2000.
  17. Morris H DeGroot. Reaching a consensus. Journal of the American Statistical association, 69(345):118–121, 1974.
  18. The spreading of misinformation online. Proceedings of the national academy of Sciences, 113(3):554–559, 2016.
  19. Echo chambers: Emotional contagion and group polarization on facebook. Scientific reports, 6(1):37825, 2016.
  20. Jan Christian Dittmer. Consensus formation under bounded confidence. Nonlinear Analysis: Theory, Methods & Applications, 47(7):4615–4621, 2001.
  21. Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982, 2020.
  22. Jeffrey L Elman. Finding structure in time. Cognitive science, 14(2):179–211, 1990.
  23. On the evolution of random graphs. Publ. math. inst. hung. acad. sci, 5(1):17–60, 1960.
  24. Generative diffusion models on graphs: Methods and applications. arXiv preprint arXiv:2302.02591, 2023.
  25. Social influence and opinions. Journal of Mathematical Sociology, 15(3-4):193–206, 1990.
  26. Graph u-nets. In international conference on machine learning, pages 2083–2092. PMLR, 2019.
  27. Neural message passing for quantum chemistry. In International conference on machine learning, pages 1263–1272. PMLR, 2017.
  28. A new model for learning in graph domains. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, pages 729–734. IEEE, 2005.
  29. Topological deep learning: Going beyond graph data. Preprint, 2023.
  30. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
  31. Jakob Hansen. Laplacians of Cellular Sheaves: Theory and Applications. PhD thesis, University of Pennsylvania, 2020.
  32. Sheaf neural networks. arXiv preprint arXiv:2012.06333, 2020.
  33. Toward a spectral theory of cellular sheaves. Journal of Applied and Computational Topology, 3:315–358, 2019.
  34. Opinion dynamics on discourse sheaves. SIAM Journal on Applied Mathematics, 81(5):2033–2060, 2021.
  35. Nonequilibrium phase transition in the coevolution of networks and opinions. Physical Review E, 74(5):056108, 2006.
  36. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. pmlr, 2015.
  37. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications, 207:117921, 2022.
  38. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pages 2323–2332. PMLR, 2018.
  39. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  40. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  41. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493, 2015.
  42. Graph neural networks for temporal graphs: State of the art, open challenges, and opportunities. arXiv preprint arXiv:2302.01018, 2023.
  43. Geodesic convolutional neural networks on riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops, pages 37–45, 2015.
  44. Geometric matrix completion with recurrent multi-graph neural networks. Advances in neural information processing systems, 30, 2017.
  45. Fake news detection on social media using geometric deep learning. arXiv preprint arXiv:1902.06673, 2019.
  46. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, volume 8, page 1, 2012.
  47. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5363–5370, 2020.
  48. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287, 2020.
  49. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017.
  50. Opinion dynamics and bounded confidence: models, analysis and simulation. Journal of Artificial Societies and Social Simulation, 2002.
  51. Caspr: Learning canonical spatiotemporal point cloud representations. Advances in neural information processing systems, 33:13688–13701, 2020.
  52. Dropedge: Towards deep graph convolutional networks on node classification. arXiv preprint arXiv:1907.10903, 2019.
  53. Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6):386, 1958.
  54. Multi-scale attributed node embedding. Journal of Complex Networks, 9(2):cnab014, 2021.
  55. The graph neural network model. IEEE transactions on neural networks, 20(1):61–80, 2008.
  56. Characterizing the latent space of molecular deep generative models with persistent homology metrics. arXiv preprint arXiv:2010.08548, 2020.
  57. Modeling relational data with graph convolutional networks. In The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15, pages 593–607. Springer, 2018.
  58. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  59. Allen Dudley Shepard. A cellular description of the derived category of a stratified space. Brown University, 1985.
  60. Graphvae: Towards generation of small graphs using variational autoencoders. In Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part I 27, pages 412–422. Springer, 2018.
  61. Vector diffusion maps and the connection laplacian. Communications on pure and applied mathematics, 65(8):1067–1144, 2012.
  62. Surfing on the neural sheaf. In NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations, 2022.
  63. Social influence analysis in large-scale networks. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 807–816, 2009.
  64. Michael Taylor. Towards a mathematical theory of influence and attitude change. Human Relations, 21(2):121–139, 1968.
  65. Loring W.. Tu. An introduction to manifolds. Springer., 2011.
  66. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
  67. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209.14734, 2022.
  68. The reduction of a graph to canonical form and the algebra which appears therein. nti, Series, 2(9):12–16, 1968.
  69. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861–6871. PMLR, 2019.
  70. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4–24, 2020.
  71. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  72. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  73. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. In 2022 IEEE International Conference on Data Mining (ICDM), pages 1287–1292. IEEE, 2022.
  74. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 974–983, 2018.
  75. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018.
  76. Graphrnn: Generating realistic graphs with deep auto-regressive models. In International conference on machine learning, pages 5708–5717. PMLR, 2018.
  77. Graph transformer networks. Advances in neural information processing systems, 32, 2019.
  78. Graphsaint: Graph sampling based inductive learning method. arXiv preprint arXiv:1907.04931, 2019.
  79. Graph neural networks: A review of methods and applications. AI open, 1:57–81, 2020.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Authors (1)

  1. Olga Zaghen 

Collections

Sign up for free to add this paper to one or more collections.

Sign Up