Pass-Bands Optimizer (PBO) for Video SNNs

• PBO is a plug-and-play module that adaptively reshapes the temporal pass-band of spiking neural networks, enabling selective enhancement of motion-relevant frequencies.
• It introduces a lightweight, parameter-efficient pre-filter before standard LIF dynamics to mitigate the low-pass bias, preserving spatial and semantic consistency.
• Empirical results demonstrate significant accuracy gains on both uni-modal and multi-modal video benchmarks with minimal additional computational overhead.

The Pass-Bands Optimizer (PBO) is a plug-and-play module designed to adaptively reshape the temporal frequency pass-band of spiking neural networks (SNNs), particularly for video-based recognition tasks. By introducing a lightweight, parameter-efficient pre-filter that precedes the standard Leaky Integrate-and-Fire (LIF) neuron dynamics, PBO overcomes the intrinsic low-pass bias of SNNs, enabling selective enhancement of motion-relevant frequency bands. This approach enables SNNs to recover motion cues while preserving semantic content and spatial boundaries, resulting in significant accuracy gains for both uni-modal and multi-modal video understanding benchmarks, with negligible computational or architectural overhead (Ye et al., 30 Jan 2026).

1. Motivation: Temporal Pass-Band Mismatch in SNNs

Standard LIF neurons in SNNs implement a first-order "leak-and-integrate" temporal dynamic that attenuates high-frequency stimuli while preserving low-frequency (near-DC) components. In the frequency domain, the LIF subthreshold dynamics act as a low-pass filter with strong DC gain and rapidly decaying magnitude at higher temporal frequencies. While non-problematic for static image benchmarks, this suppression of higher temporal frequencies becomes detrimental for video tasks, where mid- and high-frequency bands encode critical motion semantics, such as object displacement and dynamic events. Empirical power spectrum analysis on datasets like UCF101 reveals that videos exhibit substantial energy in these motion bands, which are significantly suppressed after LIF integration.

Attempts to remedy this deficit using hard high-pass filters (e.g., frame differencing) indiscriminately eliminate static content and global semantics, producing overly coarse residual signals. Thus, there exists a fundamental mismatch between the native SNN temporal transfer function and the dynamic nature of video signals, and resolving this requires more adaptive frequency shaping.

2. Mathematical Formulation

2.1 LIF Neuron Dynamics

For input $X[t] \in \mathbb{R}^d$ at time $t$, the discrete-time LIF neuron propagates as: $$U[t] = V[t-1] + T \cdot (X[t] - (V[t-1]-V_\text{reset}))$$

$S[t] = \Theta(U[t] - V_\text{th})$

$$V[t] = U[t] \cdot (1 - S[t]) + V_\text{reset} \cdot S[t]$$

with $T = 1/\tau_m$ (leak), where recentering around $V_\text{reset}$ produces an LTI system: $$V[t] = (1 - \alpha)V[t-1] + \alpha X[t], \quad \alpha = T$$ The DTFT frequency response is: $$H_\text{LIF}(e^{j\omega}) = \frac{1-\alpha}{1 - \alpha e^{-j\omega}}$$ with squared magnitude

$$|H_\text{LIF}(e^{j\omega})|^2 = \frac{(1-\alpha)^2}{1 + \alpha^2 - 2\alpha \cos\omega}$$

which sharply decays with increasing $\omega$.

2.2 PBO Pre-Filter

PBO inserts a two-tap pre-filter before LIF integration: $Y[t] = X[t] - \mu[t]\cdot X[t-1]$ where $\mu[t] \in [0,1]$ is the learnable modulation coefficient.

• Static μ: For $\mu[t] = \mu$ (constant), the frequency response is: $$|W(e^{j\omega};\mu)|^2 = 1 + \mu^2 - 2\mu \cos\omega$$ The cascade with LIF leads to: $$|G(e^{j\omega};\mu)|^2 = \frac{(1 + \mu^2 - 2\mu \cos\omega)(1 - \alpha)^2}{1 + \alpha^2 - 2\alpha\cos\omega}$$ As $\mu$ increases from $0$ to $1$, this interpolates from low-pass to a "tilted" high-pass, but without forming a true mid-band peak.
• Time-Varying μ: Setting $\mu[t] = p + A\sin(\omega t + \phi)$ with $p\in [0,1], \omega \in (0, \pi]$, and fixed small $A, \phi$ makes the filter Linear Periodically Time-Varying (LPTV), generating true mid-band emphasis by modulating the signal and injecting sidebands at $\pm\omega$. In the harmonic transfer domain, this enables translation of low-frequency energy into mid-bands, directly targeting motion-dense windows in the temporal spectrum.

3. Consistency Constraints for Semantic Preservation

While the PBO reshapes the spatiotemporal frequency content, it must preserve overall appearance and edge continuity. A regularization constraint is defined between the PBO-mixed pre-filter output $Y^{(m)}[t]$ and two references: the DC (original input, $Y^{(0)}[t]=X[t]$) and the pure high-pass ($Y^{(1)}[t]=X[t]-X[t-1]$). The consistency term is: $$C_\text{int}[t] = \mu[t]\|\hat{Y}^{(m)}[t] - \hat{Y}^{(1)}[t]\|_2^2 + (1-\mu[t])\|\hat{Y}^{(m)}[t] - \hat{Y}^{(0)}[t]\|_2^2$$ with additional gradient consistency: $$C_\text{grad}[t] = \|\nabla \hat{Y}^{(m)}[t] - \max(\|\nabla \hat{Y}^{(0)}[t]\|,\|\nabla \hat{Y}^{(1)}[t]\|)\|_1$$ The total consistency loss is $$L_\text{consist} = \sum_t [C_\text{int}[t] + C_\text{grad}[t]]$$. This regularization ensures that the filtered stream remains within the convex hull of the DC and high-pass endpoints and that boundaries are preserved.

4. Implementation and Computational Overhead

PBO is instantiated as a single pre-filter layer directly prior to the SNN input. It introduces only two learnable scalars: $p$ and $\omega$ (the latter mapped via a logistic nonlinearity onto $(0,\pi]$); amplitude $A$ and phase $\phi$ are fixed hyperparameters. For an input of shape $T\times H\times W\times C$, PBO adds exactly one additional multiplication and one subtraction per element per timestep, i.e., $\mathcal{O}(T\cdot H\cdot W\cdot C)$ total extra operations. This overhead is negligible compared to the cost of spiking embedding layers, which typically require convolutional MACs on the order of $$\mathcal{O}(T\cdot H\cdot W\cdot k^2\cdot C\cdot C_\text{out})$$. No modifications are required to the backbone, data flow, or inference engine.

5. Empirical Performance

PBO demonstrates consistent and substantial improvements across multiple video benchmarks:

Uni-Modal RGB Action Recognition:

Backbone	UCF101 (top-1 acc.)	Gain	HMDB51 (top-1 acc.)	Gain
Spikformer-2-256	46.16% → 57.71%	+11.6	—	—
SDT-V1	49.25% → 59.80%	+10.6	—	—

Multi-Modal Recognition (RGB + DVS):

Benchmark	SNN Base	Baseline	+PBO	Gain
UCF101-CEP	S-CMRL	68.13%	73.03%	+4.9
HMDB51-CEP	S-CMRL	72.33%	74.18%	+1.85
HARDVS	S-CMRL	49.10%	51.30%	+2.2

Weakly-Supervised Anomaly Detection (Color-Event UCF-Crime):

Modality	AUC	FAR
RGB	71.54%→74.14%	14.54%→10.89%
RGB + Event	70.01%→74.14%	17.89%→10.89%

Ablation studies confirm that optimal performance is achieved at a leak factor $\alpha=0.7$, and both intensity and gradient consistency losses are necessary for best results. Modulation amplitude $A=0.1$ suffices for full gain, and improvements are stable across input lengths, strides, and spatial resolutions.

6. Limitations, Extensions, and Recommendations

PBO is designed to be minimal and interpretable, relying on only two learnable parameters. More expressive parameterizations (e.g., multi-frequency modulations or multi-layer PBO cascades) could further refine the frequency response but would increase model complexity. For new datasets or very long sequences, the parameter initialization and modulation frequency may require fine-tuning.

Potential extensions include cascading PBO modules to create hierarchical band-pass filtering, integration with temporal attention or self-modal fusion in deeper SNN layers, and joint optimization with auxiliary supervisory losses reflecting motion consistency. The regularization weights and leak factor are largely robust to hyperparameter changes, with typical defaults $\alpha\approx 0.7$ and $\lambda\approx 10^{-2}$.

7. Significance and Outlook

The Pass-Bands Optimizer presents a principled solution to the inherent low-pass limitation of SNNs for dynamic video understanding. By leveraging a simple, modulated shift-and-subtract filter, PBO efficiently recovers motion frequency content while maintaining spatial and semantic consistency, requiring no changes to the underlying neural architecture. Empirical results across standard action recognition and anomaly detection benchmarks indicate substantial gains, frequently exceeding 10% accuracy improvement on RGB video tasks, and robust performance improvements on multi-modal and weakly supervised regimes. These attributes establish PBO as a practical and effective component for advancing SNN-based spatiotemporal vision systems (Ye et al., 30 Jan 2026).