Introduction to mHC-GNN Architecture

• mHC-GNN is a graph neural network architecture leveraging manifold-constrained hyper-connections to improve expressiveness and mitigate over-smoothing.
• Employs doubly stochastic constraints via Sinkhorn–Knopp normalization to maintain robust node representations across deep layers.
• Demonstrates consistent gains in node-classification benchmarks with minimal computational overhead compared to traditional GNNs.

mHC-GNN is a graph neural network architecture that employs manifold-constrained hyper-connections, adapting recent innovations from Transformer models to the graph domain. It expands each node representation into multiple parallel streams and enforces doubly stochastic constraints on the stream-mixing matrices via Sinkhorn–Knopp normalization, achieving provable improvements in mitigation of over-smoothing and expressiveness beyond the 1-Weisfeiler–Leman (1-WL) test. Empirical evaluations demonstrate consistent gains across ten benchmarks, with robustness to depth and minor computational overhead (Mishra, 5 Jan 2026).

1. Multi-Stream Expansion and Hyper-Connections

In conventional GNNs, a node $i$ is represented by a single $d$-dimensional vector $\mathbf{h}_i \in \mathbb{R}^d$. mHC-GNN generalizes this to $n$ parallel streams,

$$\mathbf{x}_i = \begin{bmatrix} \mathbf{x}_i^{(1)} \ \vdots \ \mathbf{x}_i^{(n)} \end{bmatrix} \in\mathbb{R}^{n\times d}, \quad \mathbf{x}_i^{(s)}\in\mathbb{R}^{d}$$

where the hyperparameter $n$ modulates the width–depth trade-off of the architecture.

Each layer $l$ executes two parallel paths for every node, formalized as

$$\mathbf{x}_i^{(l+1)} = H_{l,i}^{\text{res}}\,\mathbf{x}_i^{(l)} + \big(H_{l,i}^{\text{res}}\big)^{\!\top} F_{\text{GNN}}(H_{l,i}^{\text{pre}}\mathbf{x}_i^{(l)}, \{\mathbf{x}_j^{(l)}: j\in N_i\}; W^{(l)})$$

The terms are:

• $H_{l,i}^{\text{pre}}\in\mathbb{R}^{1\times n}$: aggregates streams for message-passing
• $F_{\text{GNN}}$: base GNN update (GCN, SAGE, GAT, or GIN)
• $H_{l,i}^{\text{post}}\in\mathbb{R}^{n\times 1}$: broadcasts single GNN output to streams
• $H_{l,i}^{\text{res}}\in\mathbb{R}^{n\times n}$: learnable stream residual mixing

Hyper-connections refer collectively to $$\{H_{l,i}^{\text{pre}}, H_{l,i}^{\text{post}}, H_{l,i}^{\text{res}}\}$$ and serve as the routing operator between streams.

2. Birkhoff Polytope Constraint and Sinkhorn–Knopp Normalization

mHC-GNN constrains the residual mixing matrix $H_{l,i}^{\text{res}}$ to the Birkhoff polytope: $$B_n = \left\{ H \in \mathbb{R}_+^{n\times n}: H\mathbf{1}_n = \mathbf{1}_n, \; \mathbf{1}_n^\top H = \mathbf{1}_n^\top \right\}$$ where vertices are permutation matrices.

The unconstrained score $\widehat H_{l,i}$ (from static and dynamic terms) is projected onto $B_n$ using Sinkhorn–Knopp normalization,

$M^{(0)} = \exp(\widehat H_{l,i}),$

followed by alternating normalization of rows and columns, repeated $T$ times, yielding $M^{(2T)} \approx H_{l,i}^{\text{res}} \in B_n$ as $T\to\infty$.

This constraint fosters conservation of the mean across streams, bounds spectral norm, and permits identity-like initialization for neutral scores. The manifold constraint is confirmed as essential, since removing Sinkhorn normalization causes catastrophic collapse (up to 82% degradation).

3. Over-Smoothing Mitigation and Expressiveness Analysis

Standard GNN architectures suffer from exponential contraction of inter-node differences (over-smoothing), formally,

$$\mathbb{E}\|\mathbf{h}_i^{(L)} - \mathbf{h}_j^{(L)}\| \leq C_0 (1-\gamma)^L$$

where $\gamma = 1-\lambda_2(\overline{A})$ and $L$ is the depth.

mHC-GNN, due to staggered pre-mixing, residual mixing, and message passing, provably slows the contraction rate to

$$\mathbb{E}\|\mathbf{x}_i^{(L)} - \mathbf{x}_j^{(L)}\| \leq C (1-\gamma)^{L/n}(1+\epsilon)^L$$

with $n$ streams, as substantiated via App. A.1. Setting $n=1,\epsilon=0$ returns the original GNN rate, while higher $n$ values enable much deeper architectures before collapse.

For expressiveness, standard GNNs are limited by the 1-WL test. Theoretical constructions such as non-isomorphic strongly regular graphs ($\mathrm{SRG}(16,6,2,2)$, e.g., Shrikhande vs. 4×4 lattice) cannot be distinguished by 1-WL but differ in motifs. mHC-GNN with streams, via doubly stochastic mixing, is able to allocate streams for higher-order motif aggregation, and cross-compare, thus enabling strict expressiveness gains proportional to $n$. Depth $L = O(\log N)$ suffices for global graph structure capture.

4. Empirical Evaluation Across Benchmarks

mHC-GNN is validated on ten diverse node-classification datasets:

Family	Datasets	Node Count (Range)
Small heterophilic	Texas, Wisconsin, Cornell	183–251
Medium heterophilic	Chameleon, Squirrel, Actor	2K–8K
Homophilic	Cora, CiteSeer, PubMed	2K–20K
Large-scale	ogbn-arxiv	169K

Performance metrics are reported with stream counts $n \in \{2,4,8\}$ and four base GNNs (GCN, GraphSAGE, GAT, GIN) (Mishra, 5 Jan 2026).

Depth experiments reveal that baseline GCN performance drops precipitously past 16 layers, while mHC-GNN continues to deliver robust accuracy (≥74%) up to 128 layers across Cora, CiteSeer, and PubMed. The accuracy gap exceeds 50 percentage points in ultra-deep regimes.

Accuracy on Cora vs depth:

Depth	Baseline GCN	mHC (n=2)	mHC (n=4)
2	71.7% ± 1.9	64.8% ± 3.7	64.0% ± 2.8
4	71.0% ± 2.5	72.3% ± 0.9	73.8% ± 0.9
8	71.9% ± 1.4	74.0% ± 0.9	74.5% ± 0.5
16	15.5% ± 3.9	75.5% ± 1.1	75.6% ± 0.6
32	13.5% ± 1.1	75.1% ± 0.7	75.2% ± 0.7
64	20.5% ± 5.4	75.1% ± 0.9	74.9% ± 1.7
128	21.6% ± 3.3	74.5% ± 0.8	73.4% ± 1.2

5. Computational Cost and Ablation Studies

The additional per-layer computational cost introduced by mHC-GNN is

$O(E\,d + N\,d^2 + N\,n\,d + T\,n^2\,N)$

which, for typical settings ($n=4, T=10, d\gg n$), translates to a 6–8% overhead in FLOPs and comparable wall-clock time (on a 4×A6000 Ada GPU). Memory costs scale as $O(n)$ per-node.

Ablation studies isolate contributions:

• Removing Sinkhorn constraint (“No-Sinkhorn”) leads to immediate, near-total collapse in accuracy (up to 82% loss).
• Omitting either static or dynamic routing incurs only minor accuracy loss (6–9%).
• The interaction of dynamic/static routing and manifold constraint yields robust performance and stability.

Ablation Accuracy Table (excerpt):

Config	Chameleon	Texas	Cora
Full mHC-GNN	30.09% ±1.96	58.38% ±1.48	69.72% ±2.06
Dynamic-only	30.18% ±1.94	61.08% ±5.27	68.98% ±1.36
Static-only	30.18% ±2.31	62.16% ±7.65	69.60% ±1.97
No-Sinkhorn	18.20% ±0.00	10.81% ±0.00	13.00% ±0.00

6. Context, Implications, and Applicability

mHC-GNN generalizes manifold-constrained routing—previously developed for Transformers—to graph neural networks, leveraging multi-stream expansion and doubly stochastic mixing for robust, expressive node representations. The exponential slowdown of over-smoothing, together with increased motif sensitivity, suggests new architectural avenues for deep graph learning with minimal cost and high empirical reliability. A plausible implication is that mHC-constrained designs may benefit other graph or geometric architectures beset by diffusion-induced information loss or expressiveness bottlenecks (Mishra, 5 Jan 2026).