Soft Spiking Neurons in Neuromorphic Systems

• Soft spiking neurons are computational units that replace hard thresholds with smooth, differentiable transitions, enhancing analytical tractability.
• They enable gradient-based optimization in models like LIF and QIF neurons, supporting stable training and precise control of spike timing.
• Their implementation in hardware, including organic electrochemical devices, demonstrates potential for low-power, robust neuromorphic systems.

Soft spiking neurons are event-driven computational units in which the classical sharp threshold behavior of spiking neuron models is replaced or augmented with smooth, differentiable, or otherwise "softened" transitions. The motivation for this modification encompasses the need for robust, trainable, and hardware-friendly spiking architectures that can leverage the benefits of spike-based processing—such as sparsity and asynchronous computation—while overcoming the obstacles associated with hard, non-differentiable threshold nonlinearity. These neurons are realized across diverse levels: analytic softening of classical LIF models, exact gradient-based formulations in quadratic IF neurons, fully differentiable recurrent neural event cells, mathematical softening of threshold boundaries in low-rank spiking networks, hardware substrate realizations with ion-mediated conduction, and event-driven controllers for compliant soft robots. The unifying feature is the introduction of a mechanism that removes or relaxes discontinuities at spike initiation, thereby enabling gradient-based optimization, smooth adaptation, and compatibility with a wider spectrum of machine-learning and neuromorphic methodologies.

1. Mathematical Formulations and Analytic Softening in LIF Models

The leaky integrate-and-fire (LIF) neuron is a canonical model in spiking neural networks (SNNs), described by a membrane ODE

$\tau_{\text{RC}} \frac{dv}{dt} = -v(t) + J(t)$

with reset and refractory periods. The standard LIF steady-state firing rate for input current $j > V_{\text{th}}$ is

$$r_{\text{LIF}}(j) = \left[\tau_{\text{ref}} - \tau_{\text{RC}} \ln(1 - V_{\text{th}}/j)\right]^{-1}, \quad r_{\text{LIF}}(j) = 0 \text{ otherwise}$$

However, $\partial r/\partial j$ diverges as $j \to V_{\text{th}}^{+}$, making backpropagation problematic. To rectify this, the hard $\max(x, 0)$ in the activation is replaced by a soft-plus approximation. The "soft" LIF activation uses

$$\rho_2(x) = \gamma \ln \left[1 + e^{x/\gamma}\right]$$

with smoothing parameter $\gamma>0$, yielding

$$r_{\text{soft}}(j) = \left[\tau_{\text{ref}} + \tau_{\text{RC}}\ln\left(1 + \frac{V_{\text{th}}}{\rho_2(j - V_{\text{th}})}\right)\right]^{-1}$$

This produces a bounded gradient everywhere, ensuring analytical tractability and robust training in deep networks. The choice of $\gamma$ trades off between fidelity to the original LIF response and numerical smoothness (Hunsberger et al., 2015).

2. Exact Gradient Approaches: Smooth Spike Timing in QIF Neurons

A distinct instantiation of soft spiking arises in the quadratic integrate-and-fire (QIF) model, where the membrane dynamics are governed by

$\frac{dV}{dt} = V^2 + I(t)$

The spike time is the instant when $V \to +\infty$. In this construction, all spike times are smooth (continuously differentiable) functions of the input and network parameters—for any small parameter change, spike times shift continuously rather than appearing or vanishing abruptly. Spikes can only vanish by sliding out of a fixed trial window $[0,T]$, and new spikes are "grown in" via a pseudospike construction beyond $T$. This smooth dependence supports exact gradient-based training, including the addition and removal of spikes through analytic derivatives of spike times with respect to weights, computed through closed-form relations for interspike intervals. The methodology enables stable learning in deep, recurrent, or initially silent spiking networks, circumventing the discontinuity problem of classical spiking neurons (Klos et al., 2023).

3. Differentiable Event-Based Units: Spiking Recurrent Cells

Another paradigm achieves "soft" spiking by altering RNN cell dynamics to be event-driven yet fully differentiable. The Spiking Recurrent Cell (SRC) replaces hard-spike emission with a continuous activation $s_{\text{out}}[t] = \max\{0, h[t]\}$, where $h[t]$ is a fast membrane state governed by nonlinear updates, positive feedback, and recovery processes:

• Synaptic integration: $$\tau_{\text{syn}}\,\frac{d i}{dt} = -i + W_s\,s_{\text{in}}(t)$$
• Membrane dynamics: $$\tau_h\,\frac{d h}{dt} = -h + \phi(i + r\,h + r_s\,h_s + b_h)$$
• Recovery dynamics for slow state $h_s$ In the forward pass, spikes are real-valued and continuous; in backpropagation, gradients bypass nondifferentiable units by direct substitution, facilitating training by vanilla BPTT. Empirically, deep (10–15 layer) SRC-based spiking networks match or exceed LIF SNNs on visual benchmarks, while preserving event-driven sparsity and hardware efficiency (Geeter et al., 2023).

4. Geometric Softening: Boundary Smoothing in Low-Rank Spiking Architectures

In analytic treatments of low-rank excitatory-inhibitory (EI) spiking networks, the classical spike threshold is recast as a hard boundary in the neuron's low-dimensional input-output space, demarcating sub- and supra-threshold regions. However, realistic factors such as synaptic filtering and voltage noise smooth the hard, piecewise-linear threshold into a finite-slope sigmoid. Mathematically, each hard indicator $H(F_ix + E_iy - T_i)$ is replaced by a smooth boundary

$\frac{1}{\tau_s}\sigma_\beta(F_ix + E_iy - T_i)$

where $\sigma_\beta$ is a logistic sigmoid of slope $\beta$. As the boundary softens, the network's latent dynamics become equivalent to a standard low-rank rate network. The degree of softening (regulated by synaptic time constant, noise level, and activation slope) directly influences the trade-off between coding precision, stability, and robustness to fluctuations. Soft boundaries preserve DC (difference-of-convex) function approximation capabilities inherent to these architectures (Podlaski et al., 2023).

5. Organic and Hardware-Realized Soft Spiking Neurons

Soft spiking neurons are not limited to algorithmic and simulation contexts; recent advances demonstrate their realization in hardware, particularly in the form of organic electrochemical neurons (OECNs). Constructed from printed complementary organic electrochemical transistors (OECTs), these devices utilize volumetric ion-induced conductivity modulation to achieve membrane-like integration, thresholding, and reset behaviors. The firing frequency is governed by device capacitance, input current, and feedback/reset mechanics, with the analytic firing rate given by

$$f = 1/\big((C_{\text{mem}}/I_{\text{in}})(V_T - V_{\text{rest}}) + R_{\text{reset}}C_{\text{mem}}\big)$$

OECNs exhibit soft, analog voltage spikes with sub-volt operation (V$_{\text{out}} \lesssim$ 0.6 V), low power ($P_{\text{dyn}} \lesssim 15$ nW), and direct interfacing with biological systems such as Venus flytrap closure responses. Their integration with organic electrochemical synapses (OECSs) supports short- and long-term plasticity, including symmetric STDP, and makes them suitable for soft robotics, biohybrid electronics, and in situ learning (Padinhare et al., 2024).

6. Soft Spiking Neuron Controllers in Soft Robotics

The double-threshold spiking neuron (DTS) model represents an application-driven variant, focusing on actuation in compliant, high-DOF soft robots. The DTS neuron employs two adjustable thresholds $(u_n, u_p)$, producing an output

$$o(t) = \begin{cases} +1, & u < u_n \ 0, & u_n \leq u \leq u_p \ -1, & u > u_p \end{cases}$$

with the membrane potential $u$ reset after each spike. This architecture allows the control of soft-body resonance, event-driven actuation, and the emergence of diverse gaits simply by modulating the threshold parameters. When embedded within reinforcement learning loops, these neurons enable sparse, robust, and interpretable policy representations, resulting in higher task success, reduced time-to-target, and smoother movements compared to torque-based or central pattern generator (CPG) controllers (Zhang et al., 31 Jan 2025).

7. Implications, Advantages, and Practical Considerations

Soft spiking neurons unify several desirable properties across modern SNN research and engineering:

• Enabling Differentiability: Bound and continuous gradients through spike initiations facilitate training deep, recurrent, and hardware-oriented SNNs by backpropagation or exact event-driven methods (Hunsberger et al., 2015, Klos et al., 2023, Geeter et al., 2023).
• Biological Plausibility: Synaptic filtering, noise-induced threshold jitter, and analog hardware implementations align with biological cortical dynamics, including E/I balance and irregular firing (Podlaski et al., 2023, Padinhare et al., 2024).
• Robustness and Energy Efficiency: Smoothing boundaries and introducing noise during training fosters resilience to hardware quantization, spike jitter, and analog variability, while event-driven actuation promotes low-power operation.
• Software/Hardware Co-design: The modularity of soft neuron formulations—whether via smooth activations, pseudospike extensions, or circuit-level implementation—facilitates mapping to neuromorphic hardware including OECT-based organic devices and FPGA/ASIC systems (Geeter et al., 2023, Padinhare et al., 2024).
• Broad Applicability: These neurons serve as foundational units in deep networks, compact RL-controlled actuators in robotics, and as mechanisms for interfacing with living biology.

A plausible implication is that future advances in high-fidelity neuromorphic systems, adaptive robotics, and biohybrid computation will increasingly depend on the versatility and robustness afforded by soft spiking neuron models, architectures, and hardware.