Spike Prompting for Robust GNNs

• Spike prompting is a paradigm that uses spiking neurons to create sparse, noise-robust modifications of input features in graph neural networks.
• It employs cascaded Integrate-and-Fire modules for selective atom selection and sparse feature modification, ensuring efficient adaptation.
• Empirical evaluations show a 2–5% improvement in few-shot node classification accuracy along with enhanced robustness against edge perturbations and meta-attacks.

Spike prompting refers to a prompt-learning paradigm in which spiking neuron architectures are exploited to produce sparse, noise-robust modifications of input features for adapting pre-trained models, particularly within graph neural network (GNN) contexts. By leveraging threshold-driven firing mechanisms intrinsic to spiking neurons, this approach enables selective augmentation of node features via prompt atoms, yielding compact and efficient representations. Spike prompting was first formalized in the Spiking Graph Prompt Feature (SpikingGPF) framework, which replaces conventional dense prompt computation with cascaded Integrate-and-Fire (IF) modules, providing fine-grained, sparse control over both atom selection and feature dimension modification (Jiang et al., 6 Jan 2026).

1. Motivation: Redundancy and Sensitivity in Conventional Prompting

In traditional Graph Prompt Feature (GPF) learning, a frozen, pre-trained GNN model is adapted by introducing a set of prompt atoms $$B = \{b_1, \dots, b_K\} \in \mathbb{R}^{K \times d}$$ into node features. For each node $i$ with initial feature $x_i \in \mathbb{R}^d$, the dense coefficient vector $s_i \in \Delta_{K-1}$ is computed via softmax over linear scores: $$s_{ik} = \exp(w_k \cdot x_i) / \sum_{\ell=1}^K \exp(w_\ell \cdot x_i)$$. The prompt is generated as $p_i = \sum_{k=1}^K s_{ik} b_k$, added to the node features for downstream learning. A key limitation of this paradigm is redundancy: all prompt atoms contribute to all node prompts, regardless of relevance. Additionally, dense modification of all feature dimensions heightens sensitivity to noisy or distractor features. These issues motivated the development of spiking neuron-based approaches for sparse and robust prompt selection.

2. Spiking Neuron Architecture for Prompting

SpikingGPF employs two cascaded Spiking Integrate-and-Fire (IF) modules for sparse prompt generation:

• S-learning (Sparse Atom Selection): For each node $i$ and atom $k$, a membrane potential sequence $v_{ik}^{(t)}$ evolves over $T$ time steps. The linear drive $\alpha_{ik} = w_k \cdot x_i$ is integrated as $\tilde v_{ik}^{(t)} = v_{ik}^{(t-1)} + \alpha_{ik}$, and a binary spike $h_{ik}^{(t)}$ is emitted when $\tilde v_{ik}^{(t)}$ exceeds threshold $\mu$: $h_{ik}^{(t)} = H(\tilde v_{ik}^{(t)} - \mu)$, with $H(z)$ the Heaviside function. After reset $$v_{ik}^{(t)} = \tilde v_{ik}^{(t)} - \mu h_{ik}^{(t)}$$, spikes are averaged to form $h_{ik} = (1/T)\sum_{t=1}^T h_{ik}^{(t)}$, followed by softmax to yield sparse $s_{ik}$.
• P-learning (Sparse Feature Prompting): The sparse $s_i$ weight combination of atoms $B$ is further processed via an IF network with signed firing: at each step, integrate $\tilde u_i^{(t)} = u_i^{(t-1)} + (s_i \cdot B^T)$, emit signed spikes $h_{id}^{(t)} \in \{\pm1, 0\}$ depending on threshold $\gamma$, and reset $u_i^{(t)}$ accordingly. The resulting prompt $p_i = (1/T)\sum_{t=1}^T h_i^{(t)}$ is a sparse vector, modifying only a minority of feature dimensions.

This architecture enables selection of a small subset of atoms per node, with modification restricted to few feature dimensions, facilitating both compactness and robust handling of input noise.

3. Sparse Prompt Representation and Sparse Optimization

Each node's prompt $p_i \in \mathbb{R}^d$ is ultimately represented as a sparse linear combination of prompt atoms, $p_i = \sum_{k=1}^K c_{ik} a_k$, with the coefficients $c_i$ constrained by $\|c_i\|_0 \leq S$ for some small $S \ll K$, a sparsity pattern enforced by the spiking S-learning module. The associated optimization problem is:

$$\min_{C, A} L^{down}(f_\Phi(X+P, A)), \ \textrm{subject to} \ P = C \cdot A, \ \|c_i\|_0 \leq S, \forall i,$$

or, in a relaxed form, penalizing with $\ell_1$ regularization:

$$L = L^{down}(f_\Phi(X+P, A)) + \lambda \sum_i \|c_i\|_1.$$

This approach formalizes prompt learning within a sparse representation theory framework, encouraging efficient atom usage.

4. Composite Learning Objective and Surrogate Gradients

Spike prompting via SpikingGPF is trained via minimization of a composite objective:

$$\mathcal{L} = \mathcal{L}_{task}\bigl(f_\Phi(X+P, A), Y\bigr) + \lambda \sum_{i=1}^n \|s_i\|_1 + \gamma \sum_{i=1}^n \sum_{t=1}^T \|h_i^{(t)}\|_1,$$

where $\mathcal{L}_{task}$ is a cross-entropy loss over labeled nodes $Y$, the second term penalizes atom usage (encourages sparsity in $s_i$), and the third penalizes non-zero spikes in the P-learning module. Due to the non-differentiability of the Heaviside firing function $H(\cdot)$, surrogate gradients such as piecewise-linear approximations are used for backpropagation through time. The optimization proceeds by forward computation of $s_i$ and $p_i$, feeding $X+P$ to the frozen GNN $f_\Phi$ and a trainable task head, followed by backward propagation through surrogate derivatives, updating only prompt atoms $B$, S-learning parameters $W$, and task head parameters $\theta$ (Jiang et al., 6 Jan 2026).

5. Empirical Evaluation and Performance Metrics

Experimental assessment was conducted on eleven benchmarks, including citation networks (Cora, CiteSeer, PubMed), co-purchase networks (Photo, Computers), co-author networks (CS, Physics), heterophilic web graphs (Wisconsin, Texas, Cornell), and OGBN-arxiv. Baseline comparisons included supervised GNNs (GCN, GAT), pre-train/fine-tune protocols, and a range of prompt learning methods (GPPT, GraphPrompt, All-in-One, GPF, GPF+), as well as EdgePred, SUPT, and VNT.

• Metric: Few-shot node classification accuracy (1–10 shots).
• Key findings:
  • SpikingGPF outperformed GPF/GPF+ by 2–5% in 1-shot settings across all backbone models (GraphCL, SimGRACE, GraphMAE, EdgePred).
  • Atom sparsity: Only 10–20% of atoms selected per node by tuning $\mu$ and $T$.
  • Feature-dimension sparsity: Only 20–40% of $p_i$ dimensions non-zero; classification accuracy remained high.
  • Robustness: Under random edge perturbations (20–100%) and meta-attacks (5–10%), SpikingGPF exhibited superior degradation resilience relative to dense GPF approaches.

The empirical evidence supports the efficacy and robustness of spike prompting for sparse and resilient adaptation of pre-trained GNNs.

6. Broader Implications and Future Directions

SpikingGPF is the first formal articulation of “spike prompting,” demonstrating that spiking-neuron modules can be embedded within prompt learning systems to yield ultra-sparse, noise-robust feature modifications. This architecture suggests extensibility to other domains, including:

• Vision Transformers, where spiking layers could manage selection of patch embeddings for augmentation.
• LLMs, via binary gating over embedding dimensions.
• Heterogeneous graphs, applying spiking prompts over diverse edge types.

A plausible implication is that spike prompting could facilitate energy-efficient, on-device tuning of pre-trained models when deployed on neuromorphic hardware. SpikingGPF thus initiates a new line of research into “neuromorphic prompting” methods applicable across modalities. Further investigation into architectural variants, scaling properties, and hardware realization is warranted (Jiang et al., 6 Jan 2026).