Exponential Decay Gating (EDG) in Vision GNNs

• Exponential Decay Gating (EDG) is a content-aware, numerically stable mechanism that adaptively weights graph connections based on feature similarity.
• EDG integrates into Adaptive Graph Convolution blocks to dynamically suppress semantically dissimilar features, effectively mitigating over-squashing.
• EDG enhances computational efficiency and accuracy in vision GNNs by leveraging a learnable temperature parameter with minimal extra complexity.

Exponential Decay Gating (EDG) is a content-aware, numerically stable gating mechanism designed to selectively weight long-range connections in vision graph neural networks (ViGNNs) based on feature similarity. Introduced in the context of the AdaptViG hybrid Vision GNN architecture, EDG transforms a pre-defined static scaffold of graph connections into a dynamic, adaptive structure by softly suppressing contributions from semantically dissimilar regions. This approach addresses computational bottlenecks in graph construction while enhancing feature aggregation and mitigating the over-squashing phenomenon inherent to static graphs (Munir et al., 13 Nov 2025).

1. Mathematical Formulation

Given a feature map $X \in \mathbb{R}^{B \times C \times H \times W}$ with batch size $B$, channel dimension $C$, and spatial grid $H \times W$ (nodes or patches), EDG defines the gating process as follows:

• For a center node $p$ with feature $x_p \in \mathbb{R}^C$ and a potential neighbor $n$ with feature $x_n \in \mathbb{R}^C$:
  • Feature dissimilarity:

  $$d_{pn} = \lVert x_p - x_n \rVert_1 = \sum_{c=1}^C |x_{p,c} - x_{n,c}|$$ - Exponential Decay Gate:

  $g_{pn} = \exp\left(-\frac{d_{pn}}{T}\right)$

  where $T > 0$ is a scalar "temperature" parameter enforcing how sharply the gate decays as $d_{pn}$ increases. For numerical stability, $T$ is reparameterized as $|T| + \epsilon$, $\epsilon \approx 10^{-6}$.

By construction, $g_{pn}$ attains unity for identical features and decays smoothly toward zero for increasing dissimilarity, without explicit thresholds or normalization.

2. Theoretical Motivation and Properties

EDG is motivated by the need to preserve efficient, fixed-pattern (axial, logarithmic) graph scaffolds in ViGNNs while introducing adaptivity based on content similarity. The use of the L1 distance as $d_{pn}$ yields outlier resistance and a semantically meaningful measure. The exponentially decaying gate $g_{pn}$ softly weights edges, preventing abrupt disconnections or the need for global normalization (such as softmax), and does not depend on feature magnitude. The temperature $T$ directly modulates selectivity: smaller $T$ leads to steeper decay and greater selectivity, while a larger $T$ yields more permissive connectivity.

3. Integration into Adaptive Graph Convolution

EDG is employed within the Adaptive Graph Convolution (AGC) block of AdaptViG to enable efficient and adaptive receptive field expansion. The AGC procedure comprises:

1. Local connectivity via static $K$-hop axial shifts.
2. Long-range, logarithmic ($2^i$-hop) axial connections along both spatial dimensions.
3. For each long-range neighbor, L1-based feature disparities are computed, and their influence is modulated by the corresponding exponential decay gate.
4. All aggregated contributions are combined and fused with the original feature through a convolutional block.

The method ensures content-awareness while retaining computational tractability, requiring only a small set of shift, L1 distance, and exponential operations per node.

Node Relation Type	Connection Principle	Gating Applied
Local (K-hop)	Static/dilated axial connections	No
Logarithmic (2^i-hop)	Static long-range, log-spaced	Exponential Gate

4. Hyperparameters and Parameterization

The decay temperature $T$ constitutes the sole gating hyperparameter in EDG. It is initialized to 1.0 and learned independently for each AGC layer via backpropagation, always constrained to the positive domain. No additional normalization or thresholding is required; all gates naturally reside in $(0,1]$. Ablation studies on AdaptViG-B reveal that learning $T$ confers a minor accuracy gain over fixed values (e.g., $+0.1\%$ Top-1 for learned vs fixed $T$), with converged values demonstrating stage-specific adaptivity: permissive gating in early layers ($\bar T \approx 3.93$), highly selective gating in intermediate layers ($\bar T \approx 0.68$), and moderate selectivity in later layers ($\bar T \approx 1.11$).

5. Computational Efficiency Analysis

EDG confers substantial efficiency benefits relative to traditional graph construction methods. The per-node complexity is $O(C(\log H + \log W))$ due to the logarithmic number of long-range hops and one $O(C)$ L1/exponential operation per hop, compared to $O(N)$ or $O(N^2)$ for methods relying on KNN search. Empirical benchmarks (on RTX A6000) report graph construction latency for AGC+EDG at 0.048 ms, an order of magnitude below naive KNN (0.38 ms), and comparable to static approaches (SVGA: 0.04 ms). End-to-end, AdaptViG-S using AGC/EDG achieves 42.3 ms inference latency and higher Top-1 accuracy (79.6%) compared to its KNN-based counterpart (71.2 ms, 78.9%).

6. Empirical Impact and Ablation Evidence

Ablation studies establish the effectiveness and necessity of EDG:

• Removing gating and relying solely on the static scaffold reduces Top-1 accuracy by 1.1% (82.6% → 81.5%, ImageNet-1K).
• Gating without a final Global Attention stage performs marginally worse (–0.1%).
• L1+Exp gating achieves 83.3% Top-1 (AdaptViG-B); L2+Exp and L1+Sigmoid are slightly inferior (–0.1% and –0.3%, respectively).
• Gating is necessary to alleviate over-squashing and to sustain adaptive feature propagation across the graph.
• The integration of EDG accounts for approximately 1% of the net Top-1 accuracy improvement in AdaptViG.

7. Significance and Context within Vision GNNs

EDG enables the synthesis of sparse, adaptive, and content-aware graphs from static connection patterns, permitting lightweight, accurate, and scalable ViGNNs. The mechanism requires only a minimal per-layer learnable parameter, introduces logarithmic computational overhead, and circumvents pitfalls associated with normalization or threshold selection. EDG demonstrates clear empirical advantages in both efficiency and accuracy, directly substantiated by controlled ablation studies (Munir et al., 13 Nov 2025). Its introduction in AdaptViG marks a significant advance in graph-based vision architectures, reconciling computational tractability with adaptive global context aggregation.