Pairwise Alignment Improves Graph Domain Adaptation
Abstract: Graph-based methods, pivotal for label inference over interconnected objects in many real-world applications, often encounter generalization challenges, if the graph used for model training differs significantly from the graph used for testing. This work delves into Graph Domain Adaptation (GDA) to address the unique complexities of distribution shifts over graph data, where interconnected data points experience shifts in features, labels, and in particular, connecting patterns. We propose a novel, theoretically principled method, Pairwise Alignment (Pair-Align) to counter graph structure shift by mitigating conditional structure shift (CSS) and label shift (LS). Pair-Align uses edge weights to recalibrate the influence among neighboring nodes to handle CSS and adjusts the classification loss with label weights to handle LS. Our method demonstrates superior performance in real-world applications, including node classification with region shift in social networks, and the pileup mitigation task in particle colliding experiments. For the first application, we also curate the largest dataset by far for GDA studies. Our method shows strong performance in synthetic and other existing benchmark datasets.
- Anonymous. Improved invariant learning for node-level out-of-distribution generalization on graphs. Submitted to The Twelfth International Conference on Learning Representations, 2023. under review.
- Regularized learning for domain adaptation under label shifts. International Conference on Learning Representations, 2018.
- Pileup per particle identification. Journal of High Energy Physics, 2014.
- Size-invariant graph representations for graph classification extrapolations. International Conference on Machine Learning, 2021.
- Graph domain adaptation: A generative view. arXiv preprint arXiv:2106.07482, 2021.
- Learning causally invariant representations for out-of-distribution generalization on graphs. Advances in Neural Information Processing Systems, 2022.
- Does invariant graph learning via environment augmentation learn invariance? Advances in Neural Information Processing Systems, 2023.
- Tree mover’s distance: Bridging graph metrics and stability of graph neural networks. Advances in Neural Information Processing Systems, 2022.
- Unsupervised domain adaptation via regularized conditional alignment. Proceedings of the IEEE/CVF international conference on computer vision, 2019.
- Graph transfer learning via adversarial domain adaptation with graph convolution. IEEE Transactions on Knowledge and Data Engineering, 2022.
- Contextual stochastic block models. Advances in Neural Information Processing Systems, 31, 2018.
- A closer look at distribution shifts and out-of-distribution generalization on graphs. NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
- Enhancing graph neural network-based fraud detectors against camouflaged fraudsters. Proceedings of the 29th ACM international conference on information & knowledge management, 2020.
- Domain-adversarial training of neural networks. The journal of machine learning research, 2016.
- Domain adaptation with conditional transferable components. International Conference on Machine Learning, 2016.
- Joint learning of label and environment causal independence for graph out-of-distribution generalization. Advances in Neural Information Processing Systems, 2023.
- Inductive representation learning on large graphs. Advances in Neural Information Processing Systems, 2017.
- G-mixup: Graph data augmentation for graph classification. International Conference on Machine Learning, 2022.
- Highfield, R. Large hadron collider: Thirteen ways to change the world. The Daily Telegraph. London. Retrieved, 2008.
- Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, 2020.
- Jackson, M. O. et al. Social and economic networks, volume 3. Princeton university press Princeton, 2008.
- Drugood: Out-of-distribution dataset curator and benchmark for ai-aided drug discovery–a focus on affinity prediction problems with noise annotations. Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
- Graph invariant learning with subgraph co-mixup for out-of-distribution generalization. arXiv preprint arXiv:2312.10988, 2023.
- Empowering graph representation learning with test-time graph transformation. International Conference on Learning Representations, 2022.
- Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2016.
- Wilds: A benchmark of in-the-wild distribution shifts. International Conference on Machine Learning, 2021.
- Pileup mitigation with machine learning (pumml). Journal of High Energy Physics, 2017.
- Learning invariant graph representations for out-of-distribution generalization. Advances in Neural Information Processing Systems, 2022a.
- Semi-supervised graph neural networks for pileup noise removal. The European Physics Journal C, 2022b.
- Information obfuscation of graph neural networks. International Conference on Machine Learning, 2021.
- Graph mixup with soft alignments. International Conference on Machine Learning, 2023.
- Detecting and correcting for label shift with black box predictors. International Conference on Machine Learning, 2018.
- Rethinking propagation for unsupervised graph domain adaptation. 2024.
- Structural re-weighting improves graph domain adaptation. International Conference on Machine Learning, 2023.
- Adversarial unsupervised domain adaptation with conditional and label shift: Infer, align and iterate. Proceedings of the IEEE/CVF international conference on computer vision, 2021.
- Learning transferable features with deep adaptation networks. International Conference on Machine Learning, 2015.
- Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems, 2018.
- Interpretable and generalizable graph learning via stochastic attention mechanism. International Conference on Machine Learning.
- Sa-gda: Spectral augmentation for graph domain adaptation. Proceedings of the 31st ACM International Conference on Multimedia, 2023.
- Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
- Invariant models for causal transfer learning. The Journal of Machine Learning Research, 2018.
- Adversarial deep network embedding for cross-network node classification. Proceedings of the AAAI conference on artificial intelligence, 2020a.
- Network together: Node classification via cross-network deep network embedding. IEEE Transactions on Neural Networks and Learning Systems, 2020b.
- Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(9), 2011.
- Graph neural networks in particle physics. Machine Learning: Science and Technology, 2020.
- Unleashing the power of graph data augmentation on covariate distribution shift. Advances in Neural Information Processing Systems, 2023.
- String v11: protein–protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic acids research, 2019.
- Domain adaptation with conditional distribution matching and generalized label shift. Advances in Neural Information Processing Systems, 2020.
- Arnetminer: extraction and mining of academic social networks. Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, 2008.
- Adversarial discriminative domain adaptation. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
- Graph attention networks. International Conference on Learning Representations, 2018.
- A semi-supervised graph attentive network for financial fraud detection. IEEE International Conference on Data Mining, 2019.
- Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 2020.
- Mixup for node and graph classification. Proceedings of the Web Conference, 2021.
- Understanding non-linearity in graph neural networks from the bayesian-inference perspective. Advances in Neural Information Processing Systems, 2022.
- Non-iid transfer learning on graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
- Unsupervised domain adaptive graph convolutional networks. Proceedings of The Web Conference, 2020.
- Handling distribution shifts on graphs: An invariance perspective. International Conference on Learning Representations, 2022.
- Discovering invariant rationales for graph neural networks. International Conference on Learning Representations, 2021.
- How powerful are graph neural networks? International Conference on Learning Representations, 2018.
- Learning substructure invariance for out-of-distribution molecular representations. Advances in Neural Information Processing Systems, 2022.
- From local structures to size generalization in graph neural networks. International Conference on Machine Learning, 2021.
- Deal: An unsupervised domain adaptive framework for graph-level classification. Proceedings of the 30th ACM International Conference on Multimedia, 2022.
- Coco: A coupled contrastive framework for unsupervised domain adaptive graph classification. Internationl Conference on Machine Learning, 2023.
- Graph domain adaptation via theory-grounded spectral regularization. International Conference on Learning Representations, 2023.
- Graph information bottleneck for subgraph recognition. International Conference on Learning Representations, 2020.
- Central moment discrepancy (cmd) for domain-invariant representation learning. International Conference on Learning Representations, 2016.
- Domain adaptation under target and conditional shift. International Conference on Machine Learning, 2013.
- Adversarial separation network for cross-network node classification. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021.
- Dane: Domain adaptive network embedding. IJCAI International Joint Conference on Artificial Intelligence, 2019.
- Adversarial multiple source domain adaptation. Advances in Neural Information Processing Systems, 2018.
- On learning invariant representations for domain adaptation. International Conference on Machine Learning, 2019.
- Shift-robust gnns: Overcoming the limitations of localized graph training data. Advances in Neural Information Processing Systems, 2021.
- Shift-robust node classification via graph adversarial clustering. arXiv preprint arXiv:2203.15802, 2022.
- Explaining and adapting graph conditional shift. arXiv preprint arXiv:2306.03256, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.