DEMO-Net: Degree-specific Graph Neural Networks for Node and Graph Classification

Abstract: Graph data widely exist in many high-impact applications. Inspired by the success of deep learning in grid-structured data, graph neural network models have been proposed to learn powerful node-level or graph-level representation. However, most of the existing graph neural networks suffer from the following limitations: (1) there is limited analysis regarding the graph convolution properties, such as seed-oriented, degree-aware and order-free; (2) the node's degree-specific graph structure is not explicitly expressed in graph convolution for distinguishing structure-aware node neighborhoods; (3) the theoretical explanation regarding the graph-level pooling schemes is unclear. To address these problems, we propose a generic degree-specific graph neural network named DEMO-Net motivated by Weisfeiler-Lehman graph isomorphism test that recursively identifies 1-hop neighborhood structures. In order to explicitly capture the graph topology integrated with node attributes, we argue that graph convolution should have three properties: seed-oriented, degree-aware, order-free. To this end, we propose multi-task graph convolution where each task represents node representation learning for nodes with a specific degree value, thus leading to preserving the degree-specific graph structure. In particular, we design two multi-task learning methods: degree-specific weight and hashing functions for graph convolution. In addition, we propose a novel graph-level pooling/readout scheme for learning graph representation provably lying in a degree-specific Hilbert kernel space. The experimental results on several node and graph classification benchmark data sets demonstrate the effectiveness and efficiency of our proposed DEMO-Net over state-of-the-art graph neural network models.

An Investigation of Degree-Specific Graph Neural Networks for Node and Graph Classification

The paper under discussion introduces a novel method for learning node and graph representations using a degree-specific graph neural network (GNN) model, denoted as DSGNN. The primary aim is to address the limitations of existing GNN models, which often struggle to effectively capture the nuanced structure of graph data, particularly concerning the degree-specific graph structures, or do not fully account for theoretical aspects such as seed-oriented, degree-aware, and order-free properties in graph convolution.

Key Innovations and Approaches

The authors propose a degree-specific graph neural network inspired by the Weisfeiler-Lehman (WL) graph isomorphism test. This model recursively identifies 1-hop neighborhood structures, making it uniquely positioned to handle the graph topology alongside node attributes effectively. Importantly, DSGNN aims to explicitly preserve degree-specific graph structures through its graph convolution process, which distinguishes it from many conventional GNNs that primarily focus on node proximity.

1. Multi-Task Graph Convolution: DSGNN utilizes multi-task learning to capture degree-specific structure. For each distinct degree value, a separate task is defined, allowing the model to learn distinct node representations while preserving structural information. Two methods are proposed: degree-specific weight sharing and degree-specific hashing functions.
2. Graph-Level Pooling Scheme: A novel pooling/readout scheme is introduced to learn graph representation, which lies in a degree-specific Hilbert kernel space. This theoretical underpinning ensures that graph-level representations are robust and distinguishing of structural nuances.

Experimental Evaluation

The research demonstrates the efficacy of DSGNN across several benchmark datasets for node and graph classification tasks. The experimental results highlight the superiority of DSGNN in preserving structural information specific to each degree. It outperformed several state-of-the-art graph neural networks and structure-aware embedding approaches, underscoring the methodological novelty and practicality of the proposed approach in enhancing classification accuracy.

Theoretical Contributions

A significant aspect of the paper is its contribution to the theoretical understanding of GNNs. The authors argue for the importance of properties such as seed-oriented, degree-aware, and order-free in the design of graph convolution functions. Through mathematical proofs, the paper establishes the existence of mapping functions that maintain these properties, allowing for structurally distinct graphs to be represented distinctly in the feature space.

Moreover, DSGNN's approach to graph-level pooling and its theoretical basis in the Reproducing Kernel Hilbert Space (RKHS) induced by a degree-specific Weisfeiler-Lehman kernel, add a layer of rigor and versatility, demonstrating how degree-specific structures can be preserved even in graph-level summaries. This theoretically grounded enhancement over existing GNNs could pave the way for future innovations in graph-based learning tasks.

Implications and Future Directions

The implications of this research are both practical and theoretical. Practically, DSGNN offers a robust framework for tasks requiring nuanced graph representation, such as bioinformatics, social network analysis, and financial fraud detection. Theoretically, this work prompts a reevaluation of current graph convolution methods, emphasizing the potential for multi-task learning to improve feature aggregation in neural networks.

Future directions may involve exploring the scalability of DSGNN to larger networks, the consideration of dynamic graphs, and the integration of additional layer types to capture more complex interactions within graph data. This research could also inspire new hybrid approaches that blend degree-specific insights with other graph properties to further enhance representation learning in various applied scenarios.

In summary, the paper provides a comprehensive examination and improvement of graph neural networks through a degree-specific lens, contributing an important perspective to the ongoing development of sophisticated models for node and graph classification tasks.