Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Graph Transformer-Driven Approach for Network Robustness Learning

Published 12 Jun 2023 in cs.LG, cs.AI, and cs.CR | (2306.06913v2)

Abstract: Learning and analysis of network robustness, including controllability robustness and connectivity robustness, is critical for various networked systems against attacks. Traditionally, network robustness is determined by attack simulations, which is very time-consuming and even incapable for large-scale networks. Network Robustness Learning, which is dedicated to learning network robustness with high precision and high speed, provides a powerful tool to analyze network robustness by replacing simulations. In this paper, a novel versatile and unified robustness learning approach via graph transformer (NRL-GT) is proposed, which accomplishes the task of controllability robustness learning and connectivity robustness learning from multiple aspects including robustness curve learning, overall robustness learning, and synthetic network classification. Numerous experiments show that: 1) NRL-GT is a unified learning framework for controllability robustness and connectivity robustness, demonstrating a strong generalization ability to ensure high precision when training and test sets are distributed differently; 2) Compared to the cutting-edge methods, NRL-GT can simultaneously perform network robustness learning from multiple aspects and obtains superior results in less time. NRL-GT is also able to deal with complex networks of different size with low learning error and high efficiency; 3) It is worth mentioning that the backbone of NRL-GT can serve as a transferable feature learning module for complex networks of different size and different downstream tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. X.-F. Wang, X. Li, G. R. Chen et al., “Complex network theory and its application,” Beijing: Qing Hua University Publication, 2006.
  2. Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabási, “Controllability of complex networks,” Nature, vol. 473, no. 7346, pp. 167–173, 2011.
  3. Z. Yuan, C. Zhao, Z. Di, W.-X. Wang, and Y.-C. Lai, “Exact controllability of complex networks,” Nature Communications, vol. 4, no. 1, pp. 1–9, 2013.
  4. J.-N. Wu, X. Li, and G. Chen, “Controllability of deep-coupling dynamical networks,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 67, no. 12, pp. 5211–5222, 2020.
  5. Y. Zhang and T. Zhou, “Controllability analysis for a networked dynamic system with autonomous subsystems,” IEEE Transactions on Automatic Control, vol. 62, no. 7, pp. 3408–3415, 2016.
  6. B. Hou, X. Li, and G. Chen, “Structural controllability of temporally switching networks,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 63, no. 10, pp. 1771–1781, 2016.
  7. J. Zhu, L. Xiang, Y. Yu, F. Chen, and G. Chen, “Average controllability of complex networks with laplacian dynamics,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, no. 4, pp. 1704–1714, 2021.
  8. L. Xiang, F. Chen, W. Ren, and G. Chen, “Advances in network controllability,” IEEE Circuits and Systems Magazine, vol. 19, no. 2, pp. 8–32, 2019.
  9. A. E. Motter, “Networkcontrology,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 25, no. 9, p. 097621, 2015.
  10. L. Wang, G. Chen, X. Wang, and W. K. Tang, “Controllability of networked mimo systems,” Automatica, vol. 69, pp. 405–409, 2016.
  11. L.-Z. Wang, Y.-Z. Chen, W.-X. Wang, and Y.-C. Lai, “Physical controllability of complex networks,” Scientific reports, vol. 7, no. 1, pp. 1–14, 2017.
  12. Y.-Y. Liu and A.-L. Barabási, “Control principles of complex systems,” Reviews of Modern Physics, vol. 88, no. 3, p. 035006, 2016.
  13. G. Menichetti, L. Dall’Asta, and G. Bianconi, “Network controllability is determined by the density of low in-degree and out-degree nodes,” Physical Review Letters, vol. 113, no. 7, p. 078701, 2014.
  14. M. Pósfai, Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabási, “Effect of correlations on network controllability,” Scientific Reports, vol. 3, no. 1, pp. 1–7, 2013.
  15. U. Kang and C. Faloutsos, “Beyond’caveman communities’: Hubs and spokes for graph compression and mining,” in 2011 IEEE 11th international conference on data mining.   IEEE, 2011, pp. 300–309.
  16. Y. Lou, L. Wang, and G. Chen, “Toward stronger robustness of network controllability: A snapback network model,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 65, no. 9, pp. 2983–2991, 2018.
  17. G. Chen, Y. Lou, and L. Wang, “A comparative study on controllability robustness of complex networks,” IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 66, no. 5, pp. 828–832, 2019.
  18. Y. Lou, L. Wang, and G. Chen, “Enhancing controllability robustness of-snapback networks through redirecting edges,” Research, vol. 2019, 2019.
  19. V. H. Louzada, F. Daolio, H. J. Herrmann, and M. Tomassini, “Smart rewiring for network robustness,” Journal of Complex networks, vol. 1, no. 2, pp. 150–159, 2013.
  20. L. Bai, Y.-D. Xiao, L.-L. Hou, and S.-Y. Lao, “Smart rewiring: Improving network robustness faster,” Chinese Physics Letters, vol. 32, no. 7, p. 078901, 2015.
  21. T. Nie, B. Fan, and Z. Wang, “Complexity and robustness of weighted circuit network of placement,” Physica A: Statistical Mechanics and its Applications, vol. 598, p. 127346, 2022.
  22. L. Cuadra, S. Salcedo-Sanz, J. Del Ser, S. Jiménez-Fernández, and Z. W. Geem, “A critical review of robustness in power grids using complex networks concepts,” Energies, vol. 8, no. 9, pp. 9211–9265, 2015.
  23. R. Cohen, K. Erez, D. Ben-Avraham, and S. Havlin, “Breakdown of the internet under intentional attack,” Physical Review Letters, vol. 86, no. 16, p. 3682, 2001.
  24. “A high-robustness and low-cost model for cascading failures,” Europhysics Letters, vol. 78, no. 4, p. 48001, 2007.
  25. J.-W. Wang and L.-L. Rong, “Robustness of the western united states power grid under edge attack strategies due to cascading failures,” Safety science, vol. 49, no. 6, pp. 807–812, 2011.
  26. Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabási, “Control centrality and hierarchical structure in complex networks,” PloS one, vol. 7, no. 9, p. e44459, 2012.
  27. B. Shargel, H. Sayama, I. R. Epstein, and Y. Bar-Yam, “Optimization of robustness and connectivity in complex networks,” Physical Review Letters, vol. 90, no. 6, p. 068701, 2003.
  28. G. Chen, “Controllability robustness of complex networks,” Journal of Automation and Intelligence, vol. 1, no. 1, p. 100004, 2022.
  29. C. M. Schneider, A. A. Moreira, J. S. Andrade Jr, S. Havlin, and H. J. Herrmann, “Mitigation of malicious attacks on networks,” Proceedings of the National Academy of Sciences, vol. 108, no. 10, pp. 3838–3841, 2011.
  30. Y. Lou, L. Wang, K.-F. Tsang, and G. Chen, “Towards optimal robustness of network controllability: An empirical necessary condition,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 67, no. 9, pp. 3163–3174, 2020.
  31. Y. Lou, Y. He, L. Wang, and G. Chen, “Predicting network controllability robustness: A convolutional neural network approach,” IEEE Transactions on Cybernetics, vol. 52, no. 5, pp. 4052–4063, 2022.
  32. Y. Lou, Y. He, L. Wang, K. F. Tsang, and G. Chen, “Knowledge-based prediction of network controllability robustness,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 10, pp. 5739–5750, 2022.
  33. Y. Lou, R. Wu, J. Li, L. Wang, and G. Chen, “A convolutional neural network approach to predicting network connectedness robustness,” IEEE Transactions on Network Science and Engineering, vol. 8, no. 4, pp. 3209–3219, 2021.
  34. Y. Lou, R. Wu, J. Li, L. Wang, C.-B. Tang, and G. Chen, “Classification-based prediction of network connectivity robustness,” Neural Networks, vol. 157, pp. 136–146, 2023.
  35. Y. Lou, R. Wu, J. Li, L. Wang, X. Li, and G. Chen, “A learning convolutional neural network approach for network robustness prediction,” IEEE Transactions on Cybernetics, 2022, doi:10.1109/TCYB.2022.3207878.
  36. M. Niepert, M. Ahmed, and K. Kutzkov, “Learning convolutional neural networks for graphs,” in International Conference on Machine Learning.   PMLR, 2016, pp. 2014–2023.
  37. Y. Liu, X. Yang, S. Zhou, and X. Liu, “Simple contrastive graph clustering,” IEEE Transactions on Neural Networks and Learning Systems, 2023.
  38. Y. Liu, J. Xia, S. Zhou, S. Wang, X. Guo, X. Yang, K. Liang, W. Tu, Z. S. Li, and X. Liu, “A survey of deep graph clustering: Taxonomy, challenge, and application,” arXiv preprint arXiv:2211.12875, 2022.
  39. Y. Liu, K. Liang, J. Xia, S. Zhou, X. Yang, , X. Liu, and Z. S. Li, “Dink-net: Neural clustering on large graphs,” in Proc. of ICML, 2023.
  40. X. He, B. Wang, Y. Hu, J. Gao, Y. Sun, and B. Yin, “Parallelly adaptive graph convolutional clustering model,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  41. L. Wu, Y. Chen, K. Shen, X. Guo, H. Gao, S. Li, J. Pei, B. Long et al., “Graph neural networks for natural language processing: A survey,” Foundations and Trends® in Machine Learning, vol. 16, no. 2, pp. 119–328, 2023.
  42. S. Wu, F. Sun, W. Zhang, X. Xie, and B. Cui, “Graph neural networks in recommender systems: a survey,” ACM Computing Surveys, vol. 55, no. 5, pp. 1–37, 2022.
  43. J. Yang, J. Lu, S. Lee, D. Batra, and D. Parikh, “Graph r-cnn for scene graph generation,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 670–685.
  44. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  45. J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in International conference on machine learning.   PMLR, 2017, pp. 1263–1272.
  46. J. Zhu, J. Li, M. Zhu, L. Qian, M. Zhang, and G. Zhou, “Modeling graph structure in transformer for better amr-to-text generation,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 5459–5468.
  47. D. Q. Nguyen, T. D. Nguyen, and D. Phung, “Universal graph transformer self-attention networks,” in Companion Proceedings of the Web Conference 2022, 2022, pp. 193–196.
  48. V. P. Dwivedi and X. Bresson, “A generalization of transformer networks to graphs,” AAAI Workshop on Deep Learning on Graphs: Methods and Applications, 2021.
  49. W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances in Neural Information Processing Systems, 2017, pp. 1025–1035.
  50. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in International Conference on Learning Representations, 2017.
  51. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in International Conference on Learning Representations, 2018.
  52. N. Shao, Y. Cui, T. Liu, S. Wang, and G. Hu, “Is graph structure necessary for multi-hop question answering?” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 7187–7192.
  53. K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks, vol. 2, no. 5, pp. 359–366, 1989.
  54. Z. Wu, P. Jain, M. Wright, A. Mirhoseini, J. E. Gonzalez, and I. Stoica, “Representing long-range context for graph neural networks with global attention,” Advances in Neural Information Processing Systems, vol. 34, pp. 13 266–13 279, 2021.
  55. Z. Chen, V. Badrinarayanan, C.-Y. Lee, and A. Rabinovich, “Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,” in International conference on machine learning.   PMLR, 2018, pp. 794–803.
  56. P. Erdős and A. Rényi, “On the strength of connectedness of a random graph,” Acta Mathematica Hungarica, vol. 12, no. 1, pp. 261–267, 1961.
  57. K.-I. Goh, B. Kahng, and D. Kim, “Universal behavior of load distribution in scale-free networks,” Physical Review Letters, vol. 87, no. 27, p. 278701, 2001.
  58. M. E. Newman and D. J. Watts, “Renormalization group analysis of the small-world network model,” Physics Letters A, vol. 263, no. 4, pp. 341–346, 1999.
  59. A.-L. Barabási and R. Albert, “Emergence of scaling in random networks,” science, vol. 286, no. 5439, pp. 509–512, 1999.
  60. N. Perra and S. Fortunato, “Spectral centrality measures in complex networks,” Physical Review E, vol. 78, no. 3, p. 036107, 2008.
  61. Y. Feng, H. You, Z. Zhang, R. Ji, and Y. Gao, “Hypergraph neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 3558–3565.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Authors (4)

  1. Yu Zhang 
  2. Jia Li 
  3. Jie Ding 
  4. Xiang Li 

Collections

Sign up for free to add this paper to one or more collections.

Sign Up