Papers
Topics
Authors
Recent
Search
2000 character limit reached

Valid Conformal Prediction for Dynamic GNNs

Published 29 May 2024 in stat.ML and cs.LG | (2405.19230v2)

Abstract: Dynamic graphs provide a flexible data abstraction for modelling many sorts of real-world systems, such as transport, trade, and social networks. Graph neural networks (GNNs) are powerful tools allowing for different kinds of prediction and inference on these systems, but getting a handle on uncertainty, especially in dynamic settings, is a challenging problem. In this work we propose to use a dynamic graph representation known in the tensor literature as the unfolding, to achieve valid prediction sets via conformal prediction. This representation, a simple graph, can be input to any standard GNN and does not require any modification to existing GNN architectures or conformal prediction routines. One of our key contributions is a careful mathematical consideration of the different inference scenarios which can arise in a dynamic graph modelling context. For a range of practically relevant cases, we obtain valid prediction sets with almost no assumptions, even dispensing with exchangeability. In a more challenging scenario, which we call the semi-inductive regime, we achieve valid prediction under stronger assumptions, akin to stationarity. We provide real data examples demonstrating validity, showing improved accuracy over baselines, and sign-posting different failure modes which can occur when those assumptions are violated.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589, 2021.
  2. Convolutional networks on graphs for learning molecular fingerprints. Advances in neural information processing systems, 28, 2015.
  3. Neural message passing for quantum chemistry. In International conference on machine learning, pages 1263–1272. PMLR, 2017.
  4. A deep learning approach to antibiotic discovery. Cell, 180(4):688–702, 2020.
  5. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947, 2020.
  6. Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In Proceedings of the 2018 world wide web conference, pages 1063–1072, 2018.
  7. Graph neural networks for natural language processing: A survey. Foundations and Trends® in Machine Learning, 16(2):119–328, 2023.
  8. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263, 2017.
  9. Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 55(5):1–37, 2022.
  10. Lignn: Graph neural networks at linkedin. arXiv preprint arXiv:2402.11139, 2024.
  11. Eta prediction with graph neural networks in google maps. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 3767–3776, 2021.
  12. Advancing mathematics by guiding human intuition with ai. Nature, 600(7887):70–74, 2021.
  13. Open graph benchmark. https://ogb.stanford.edu/. Accessed: 2024-05-22.
  14. William L Hamilton. Graph representation learning. Morgan & Claypool Publishers, 2020.
  15. A gentle introduction to graph neural networks. Distill, 2021. https://distill.pub/2021/gnn-intro.
  16. Pyg documentation. https://pytorch-geometric.readthedocs.io/en/latest/. Accessed: 2024-05-22.
  17. Algorithmic learning in a random world, volume 29. Springer, 2005.
  18. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
  19. Distribution-free prediction sets. Journal of the American Statistical Association, 108(501):278–287, 2013.
  20. Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):71–96, 2014.
  21. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018.
  22. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10(2):455–482, 2021.
  23. Conformal prediction under covariate shift. Advances in neural information processing systems, 32, 2019.
  24. Conformal prediction beyond exchangeability. The Annals of Statistics, 51(2):816–845, 2023.
  25. Conformal prediction with conditional guarantees. arXiv preprint arXiv:2305.12616, 2023.
  26. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 33:3581–3591, 2020.
  27. Temporal graph benchmark for machine learning on temporal graphs. Advances in Neural Information Processing Systems, 36, 2024.
  28. Jase Clarkson. Distribution free prediction sets for node classification. In International Conference on Machine Learning, pages 6268–6278. PMLR, 2023.
  29. Conformal inductive graph neural networks. In The Twelfth International Conference on Learning Representations, 2023.
  30. Graph neural networks for intrusion detection: A survey. IEEE Access, 2023.
  31. Illuminati: Towards explaining graph neural networks for cybersecurity analysis. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pages 74–89. IEEE, 2022.
  32. Towards next-generation cybersecurity with graph ai. ACM SIGOPS Operating Systems Review, 55(1):61–67, 2021.
  33. Building and using a knowledge graph to combat human trafficking. In The Semantic Web-ISWC 2015: 14th International Semantic Web Conference, Bethlehem, PA, USA, October 11-15, 2015, Proceedings, Part II 14, pages 205–221. Springer, 2015.
  34. T-net: Weakly supervised graph learning for combatting human trafficking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38(20), pages 22276–22284, 2024.
  35. Fake news detection: A survey of graph neural network methods. Applied Soft Computing, page 110235, 2023.
  36. Deeptrace: Learning to optimize contact tracing in epidemic networks with graph neural networks. arXiv preprint arXiv:2211.00880, 2022.
  37. Heterogeneous similarity graph neural network on electronic health records. In 2020 IEEE international conference on big data (big data), pages 1196–1205. IEEE, 2020.
  38. Graph neural networks for clinical risk prediction based on electronic health records: A survey. Journal of Biomedical Informatics, page 104616, 2024.
  39. Tensor svd: Statistical and computational limits. IEEE Transactions on Information Theory, 64(11):7311–7338, 2018.
  40. The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics. The Annals of Statistics, 47(5):2405 – 2439, 2019.
  41. Entrywise eigenvector analysis of random matrices with low expected rank. Annals of statistics, 48(3):1452, 2020.
  42. A statistical interpretation of spectral embedding: the generalised random dot product graph. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(4):1446–1473, 2022.
  43. The multilayer random dot product graph. arXiv preprint arXiv:2007.10455, 2020.
  44. Spectral embedding for dynamic networks with stability guarantees. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 10158–10170. Curran Associates, Inc., 2021.
  45. Estimating higher-order mixed memberships via the ℓ2,∞subscriptℓ2\ell_{2,\infty}roman_ℓ start_POSTSUBSCRIPT 2 , ∞ end_POSTSUBSCRIPT tensor perturbation bound. arXiv preprint arXiv:2212.08642, 2022.
  46. A multilinear singular value decomposition. SIAM journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000.
  47. A simple and powerful framework for stable dynamic network embedding. arXiv preprint arXiv:2311.09251, 2023.
  48. Multilayer random dot product graphs: Estimation and online change point detection. arXiv preprint arXiv:2306.15286, 2023.
  49. Detecting communities and their evolutions in dynamic social networks—a bayesian approach. Machine learning, 82:157–189, 2011.
  50. Dynamic stochastic blockmodels for time-evolving social networks. IEEE Journal of Selected Topics in Signal Processing, 8(4):552–562, 2014.
  51. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
  52. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  53. High-resolution measurements of face-to-face contact patterns in a primary school. PloS one, 6(8):e23176, 2011.
  54. Crowdsourced air traffic data from The OpenSky Network 2020 [CC-BY], January 2022.
  55. Rethinking agricultural trade relationships in an era of globalization. BioScience, 65(3):275–289, 2015.
  56. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
  57. Uncertainty quantification over graph with conformalized graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.
  58. Vladimir Vovk. Conditional validity of inductive conformal predictors. In Asian conference on machine learning, pages 475–490. PMLR, 2012.
  59. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 5 tweets with 39 likes about this paper.

Sign Up for Free