Spiking Boltzmann Machine Ansatz

• Spiking Boltzmann Machine ansatz is a framework that maps Boltzmann machine dynamics onto networks of stochastic spiking neurons to emulate binary state sampling.
• It leverages biophysical neuron models with digital and analog implementations to perform MCMC sampling using controlled neural noise and clocked update protocols.
• Synaptic learning is integrated via spike-timing dependent plasticity and contrastive divergence, enabling robust and energy-efficient inference on neuromorphic hardware.

The spiking Boltzmann machine (SBM) ansatz refers to a class of neuronal network constructions that realize Boltzmann machine or @@@@1@@@@ (RBM) dynamics in networks of stochastic spiking neurons, typically by leveraging biophysical spiking models and implementing Markov Chain Monte Carlo (MCMC) sampling via neural noise. This approach encompasses both hardware realizations and biologically-inspired synaptic learning, mapping the theory of statistical mechanics–based neural networks onto practical, asynchronous, event-driven or clocked spiking architectures. Core implementations include digital and analog I&F neurons, stochastic bandgap neurons, and dynamic Boltzmann machines with local spike-timing dependent plasticity as a learning rule.

1. Core Principles of the Spiking Boltzmann Machine Ansatz

The fundamental principle underlying the SBM ansatz is the emulation of the Boltzmann distribution over binary (or quantized) states in a network of spiking neurons. Each neuron operates as a stochastic binary unit: its occasional spike event within a defined time window is interpreted as a sample of a Bernoulli random variable. Collective network states thereby embed the energy landscape and statistical structure associated with the corresponding Boltzmann or RBM model.

A common core neuron model is the noisy leaky integrate-and-fire (I&F) unit, in either analog or digital form. At each discrete timestep $t$, for neuron $j$:

$$V_j(t) = V_j(t-1) + \sum_{i=1}^N x_i(t) w_{ij} - \lambda_j(t)$$

where $x_i(t)$ encodes unitary spikes, $w_{ij}$ is an integer synaptic weight, and $\lambda_j(t)$ is a stochastic leak signal. Stochastic thresholding is introduced by comparing $V_j(t)$ to a noisy threshold $\alpha_j(t) = V_t + \Delta_j(t)$ with $\Delta_j$ sampled from a discrete uniform distribution. The network’s state is updated in parallel synchronous “steps” (discrete windows or clock-cycles), ensuring alignment with Gibbs sampling protocols (Das et al., 2015).

Poisson or white-noise current inputs, as in the biophysical LIF model, provide an effective “temperature” parameter, tuning the steepness of the logistic transfer function and hence the stochasticity of the unit’s binary decision (Merolla et al., 2010).

The conditional spike probability over a window ($T_w$ ticks) is:

$$P\{\text{spike in window}\} = 1 - [1 - p_\tau(v)]^{T_w}$$

which can be tuned to match the required nonlinear sigmoid for Boltzmann sampling (Das et al., 2015, Merolla et al., 2010).

2. Network Architectures and Parameter Mapping

The SBM architecture encodes an RBM (visible and hidden layers) or fully recurrent Boltzmann machine structure by mapping each logical unit onto a spiking neuron. The synaptic weight matrix is stored as integer-valued registers (for digital hardware), while customized spike-train generators or quantized offsets implement biases. Full bipartite connectivity is realized via programmable synaptic crossbars (Das et al., 2015, Neftci et al., 2013).

In models such as the Dynamic Boltzmann Machine (DyBM), history is explicitly modelled via FIFO queues and eligibility traces, capturing a time-unfolded, infinite-layer structure that yields exact and efficient stochastic time-series inference (Osogami et al., 2015, Osogami, 2016). Weights are parameterized over lags and delays:

$$W_{i,j}^{[\delta]} = \overline{W}_{i,j}^{[\delta]} + \overline{W}_{j,i}^{[-\delta]} \qquad (\delta > 0)$$

with decay structured for tractable, local learning.

Table: Representative SBM architectures

Architecture	Neuron Model	Weight Storage
TrueNorth RBM	Digital I&F, stochastic	Integer crossbar
Merolla SBM	Analog LIF, rhythmic clock	Analog
DyBM	FIFO spikes, eligibility	Exponential lag

3. Sampling and Gibbs Dynamics

The SBM realizes Gibbs sampling by clocked or event-driven updates. In digital neuromorphic platforms, each window of ticks accumulates synaptic drive, applies stochastic leak and thresholding, and emits a spike/non-spike verdict. The outcome is sampled as:

$$P(v_j = 1 | \mathbf{v}_{\setminus j}) = \sigma \left(\sum_i w_{ij} v_i + b_j\right)$$

The required sigmoid nonlinearity is achieved by adjusting the scale factor(s) of digital weights and the magnitude of stochastic noise sources. For instance, by scaling all $w_{ij}, b_j$ by $s$, and selecting leak/thresh parameters proportional to $s$, the effective window-sampled spike probability converges to the logistic function (Das et al., 2015):

$$P_{\rm window}(v) \approx \frac{1}{1 + \exp \left(-\frac{v}{s}\right)}$$

In event-driven biophysical hardware, stochasticity is realized via noise in the input currents; the spike probability across a window matches the desired MCMC kernel (Neftci et al., 2013, Merolla et al., 2010).

4. Synaptic Learning: STDP and Contrastive Divergence

SBMs can employ both offline and online learning. In offline RBM-based SBM, parameters are trained (e.g., by contrastive divergence) in floating-point software and quantized to hardware. The core update is:

$$\Delta w_{ij} = \eta \left( \langle v_i h_j \rangle_{\rm data} - \langle v_i h_j \rangle_{\rm model} \right)$$

In online variants, learning is implemented by spike-timing dependent plasticity (STDP) rules. In the DyBM, the learning rule is derived directly from the gradient of the log-likelihood and decomposes into eligibility traces for long-term potentiation (LTP) and long-term depression (LTD), e.g. (Osogami et al., 2015, Osogami, 2016):

$$u_{i,j,k} \leftarrow u_{i,j,k} + \eta \tau^{-1} \alpha_{i,j,k}[x_j^{(t)}-E_j]$$

$$v_{i,j,l} \leftarrow v_{i,j,l} - \eta \tau^{-1}\left( \beta_{i,j,l}[x_j^{(t)}-E_j] + \gamma_{j,l}[x_i^{(t)}-E_i] \right)$$

In neuromorphic devices, event-driven contrastive divergence (eCD) uses a global modulation signal to switch between LTP (data phase) and LTD (model phase), synchronizing learning with the sampling schedule (Neftci et al., 2013).

5. Hardware Realization and Implementation Details

SBM ansatz implementations extend from custom digital VLSI platforms to analog neuromorphic chips. For example, on the IBM TrueNorth substrate (Das et al., 2015):

• All dynamics are globally clocked (e.g., 1 ms/tick).
• On-chip 24-bit LFSRs provide random sources for leak and threshold noise.
• Entire RBM samples can be drawn in 1–16 μs, consuming tens of nJ, enabling large-scale, ultra-low-power sampling.

In analog platforms (e.g., Merolla et al.), clocked global inhibition is used to synchronize discrete update windows; currents injected by Poisson sources tune the effective MCMC temperature (Merolla et al., 2010). Digital and analog architectures both feature local, parallel update semantics and can scale to high neuron/synapse counts with sparse or quantized weights.

Table: Hardware-specific SBM characteristics

Platform	Update	Stochasticity Source	Energy/sample
TrueNorth	Synchronous, digital	LFSR-leak/thresh	10–30 nJ
LIF analog (Merolla)	Clocked rhythm	Poisson current	N/A
FPGA DyBM	Asynchronous	Quantized eligibility	Model-dependent

6. Performance and Benchmark Results

In the TrueNorth RBM, MNIST test accuracy (784–500–10) was ≃90 % for both spiking and ideal samplers, even with $T_w=1$ (sampling window size one tick). Increasing the accuracy of the digital sigmoid, via higher threshold noise or leak amplitude, improved generalization on corrupted data. Generative sampling quality for a small RBM (3 visible, 2 hidden) was quantified by Kullback-Leibler divergence:

• Ideal Gibbs: KL ≈ $6\times 10^{-5}$
• Digital, zero-leak: KL ≈ 0.02
• Digital, nonzero-leak: KL ≈ 0.008

indicating that a small stochastic leak greatly improves the fidelity of hardware sampling (Das et al., 2015).

Event-driven learning with STDP in RBM configurations achieved ∼92% MNIST classification accuracy, and demonstrated robust performance under weight quantization and noise (Neftci et al., 2013).

In DyBM experiments, the Gaussian DyBM with multi-scale eligibility traces improved one-step MSE by up to 20% versus finite-lag VAR for sequence modeling, at comparable computational cost (Osogami, 2016).

7. Theoretical Context and Extensions

The SBM ansatz extends the statistical mechanics analogy from abstract two-state neural networks to spiking neural substrates, preserving key features such as energy-based dynamics, MCMC sampling, and tractable learning. Incorporation of eligibility traces and spike-timing structure introduces the capability for rich, compositional time-series modeling, with exact conditional inference remaining feasible due to the architectural constraints (e.g., absence of coupling among current-layer units in DyBM).

Beyond the standard RBM, the SBM framework directly generalizes to:

• Deep Boltzmann architectures (by stacking spiking layers gated by rhythmic inhibition)
• On-chip learning via local, continuous-time STDP rules
• Extensions to real-valued variables (Gaussian DyBM) and structured temporal forecasting (Osogami et al., 2015, Osogami, 2016, Merolla et al., 2010)

This class of constructions enables practical neuromorphic embeddings of probabilistic generative models, with asynchronous, distributed, and energy-efficient learning and inference dynamics.

---

References:

• "Gibbs Sampling with Low-Power Spiking Digital Neurons" (Das et al., 2015)
• "The thermodynamic temperature of a rhythmic spiking network" (Merolla et al., 2010)
• "Learning dynamic Boltzmann machines with spike-timing dependent plasticity" (Osogami et al., 2015)
• "Learning binary or real-valued time-series via spike-timing dependent plasticity" (Osogami, 2016)
• "Event-Driven Contrastive Divergence for Spiking Neuromorphic Systems" (Neftci et al., 2013)