Temporal Mamba: Efficient SSM for Long-Range Modeling

• Temporal Mamba is a class of state-space model neural modules that use input-adaptive discretization to capture long-range dependencies in sequential data.
• It integrates structured state space recurrences with context-dependent parameterization, enabling linear-complexity modeling for high-dimensional sequences.
• TMamba variants combine multiple scan methods, dilated convolutions, and attention mechanisms to outperform Transformer and CNN baselines in diverse applications.

Temporal Mamba (TMamba) refers to a class of state space model (SSM)-driven neural modules designed to efficiently capture long-range temporal dependencies in sequential data. TMamba blocks are formulated as parameter-efficient, input-adaptive discrete-time SSMs, often serving as plug-and-play alternatives to transformer layers or temporal convolutions in diverse domains including computer vision, time series analysis, sequential decision-making, and robotics. The core innovation across TMamba variants is the integration of structured state space recurrence with context-dependent parameterization, enabling robust, linear-complexity modeling of temporal dynamics in long, high-dimensional sequences.

1. Mathematical Foundations of Temporal Mamba Blocks

TMamba blocks are rooted in the linear time-invariant (LTI) state space model, formulated as

$h'(t) = A\,h(t) + B\,x(t), \qquad y(t) = C\,h(t)$

with $h(t)\in\mathbb R^N$ the hidden state, $x(t)\in\mathbb R^D$ the input, and $y(t)\in\mathbb R^O$ the output. Discretization via zero-order hold (ZOH) yields

$$h_k = \bar{A} h_{k-1} + \bar{B} x_k, \quad y_k = C h_k$$

where $\bar{A} = \exp(\Delta A)$ and $$\bar{B} = (\Delta A)^{-1}[\exp(\Delta A) - I]\Delta B$$. In Mamba, select parameters ($\Delta$, $B$, $C$) are made input-dependent via small neural networks, introducing dynamic, context-aware evolution of the hidden state ("selective SSM"). A convolutional form permits parallelized evaluation during training: $$y = x * \bar{K},\quad \bar{K}_k = C \bar{A}^{k-1} \bar{B}$$ Bidirectional and multi-directional scans are also used, as in BiMamba and multi-scan TMamba blocks (Luo et al., 2024, Gong et al., 14 Jan 2025).

2. Architectural Variants and Integration Strategies

TMamba has been instantiated in several architectural templates, with consistent attention to input preprocessing, hierarchical stacking, and output fusion:

• Temporal Difference Mamba (TD-Mamba): Enhances local temporal variation via a central-difference 3D convolution, then applies bidirectional Mamba state space layers and squeeze-and-excitation channel attention. Integrated within a SlowFast dual-stream architecture for multi-scale video feature extraction (Luo et al., 2024).
• Multi-directional/Scan TMamba: Employs multiple scan orders (e.g., THW, TWH, HWT, WHT plus directionality), running parallel SSMs over each scan and aggregating the outputs. Used for long-range modeling of video tokens in audio-visual segmentation (Gong et al., 14 Jan 2025).
• Dilated TMamba: Integrates dilated convolutions and scatter-based temporal subdivision to expand the receptive field efficiently without sacrificing linear-time complexity, as in temporal action detection for untrimmed videos (Sinha et al., 10 Jan 2025).
• Channel-Independent/Twin TMamba: In time series, stacking multiple TMamba blocks, possibly with parallel ("twin") SSM modules per block, yields multi-scale or hierarchical representations (Wu et al., 2024).
• Conditionally Modulated TMamba: Incorporates external temporal condition signals (e.g., from audio or video) via FiLM-like scale and shift modulation of SSM parameters, ensuring frame-wise alignment in generative models (Nguyen et al., 14 Oct 2025).

The table below summarizes the main architectural contexts for recent TMamba variants:

Application Domain	TMamba Variant	Core Components	Reference
rPPG Measurement	TD-Mamba (PhysMamba)	TDC + BiMamba + CA + SlowFast	(Luo et al., 2024)
Audio-Visual Segm.	Multi-scan TMamba	3D conv + 8 SSM scans	(Gong et al., 14 Jan 2025)
Time Series	Stacked/Twin TMamba	Cascade + residual + dual SSMs	(Wu et al., 2024)
Motion Generation	Conditionally Modulated	FiLM-modded SSM + AdaLN	(Nguyen et al., 14 Oct 2025)
Tracking, EEG, TAD	1D Autoregressive/Dilated	Sliding window SSM, dilated-conv + SSM fusion	(Xie et al., 2024, Sinha et al., 10 Jan 2025)

3. Computational Complexity and Efficiency Analysis

TMamba achieves linear complexity with respect to sequence length $T$ and input dimension $D$:

• Per-block cost: $O(T \cdot D)$ for temporal convolutions or recurrences, $O(K \cdot T \cdot C)$ when using multiple scan orders (e.g., $K=8$).
• Parameter efficiency: Compared to Transformer-based architectures, TMamba models are typically 5–20× smaller; for example, PhysMamba has 0.56M parameters vs. 7.4M in PhysFormer (Luo et al., 2024).
• GPU memory and runtime: TMamba modules enable training and inference on longer sequences (e.g., hour-long videos (Sinha et al., 10 Jan 2025), full-length EEG traces (Yang et al., 2024)) without quadratic scaling or OOM failures.

This efficiency enables TMamba blocks to be deployed in real-time applications and on resource-constrained hardware, sustaining accuracy on both short and very long sequences.

4. Empirical Performance Across Domains

Empirical studies consistently report that TMamba blocks outperform or match state-of-the-art Transformer/CNN baselines in diverse tasks:

• Remote Physiological Measurement: TD-Mamba achieves MAE=0.25 bpm, RMSE=0.40 bpm, $\rho=0.99$ on the PURE dataset, outperforming previous CNN/Transformer models with orders of magnitude fewer parameters (Luo et al., 2024).
• Audio-Visual Segmentation: Multi-scale TMamba blocks yield $M_J=68.6$, $M_F=78.8$ on AVSBench-object (best prior $M_J\approx 64.2$), with reduced GPU memory and 2× faster inference (Gong et al., 14 Jan 2025).
• Long-term Forecasting: Stacked/twin TMamba achieves lowest average MSE/MAE on 13 public benchmarks, with up to 10% lower error than iTransformer, especially prominent on univariate/periodic data (Wang et al., 2024, Wu et al., 2024, Cai et al., 2024).
• Human Motion Generation: Temporally Conditional TMamba improves alignment metrics (e.g., Beat Alignment Score from 0.24 to 0.28) and kinematic/MPJPE errors in music-to-dance and ego-to-motion tasks, outperforming cross-attention and vanilla Mamba (Nguyen et al., 14 Oct 2025).
• Temporal Action Detection: MS-Temba with dilated TMamba achieves mAP=34.9% on Toyota Smarthome Untrimmed, with 88.5% reduction in parameter count and 90.9% cut in FLOPs vs. Transformer-based TAD models (Sinha et al., 10 Jan 2025).
• EEG-based MI Classification: Temporal Mamba encoder raises accuracy by 3–7% absolute over ConvNets/EEGNet/transformers on BCI IV-2a (Yang et al., 2024).

Ablation studies consistently indicate that each architectural enhancement (temporal difference front-ends, bidirectional/scan SSM, channel-attention, multi-scale fusion) yields measurable gains, and omitting TMamba blocks reverts performance toward baseline levels.

5. Advanced Techniques and Training Strategies

The TMamba literature highlights several advanced practices:

• Scan Permutation and Robustness: Randomized scan order training and variable-aware scan decoding are used to prevent channel-order sensitivity, with variable permutation training often yielding quantifiable improvements in forecasting error (Cai et al., 2024).
• Selective Parameter Dropout: TMamba blocks may include dropout applied to the input-conditioned “selective” parameters for regularization, reducing overfitting in high-dimensional sequence regimes.
• Transfer and Foundation Models: TMamba encoders serve as backbone modules in foundation models (e.g., TSMamba) supporting zero-shot transfer via two-stage training (autoregressive patch-prediction followed by full prediction head fine-tuning), with channel-compressed attention adapters for multivariate data (Ma et al., 2024).
• Condition-aware Recurrence: In conditional generation, TMamba parameters (e.g., selection matrices $B,C$) are modulated by external context, enabling frame-wise autoregressive alignment with conditioning signals (FiLM-style modulation) (Nguyen et al., 14 Oct 2025).

6. Applications and Extensions

TMamba variants have demonstrated strong utility in areas where linear scalability and robust temporal modeling are critical:

• Vision: Video-based object and action segmentation (Gong et al., 14 Jan 2025), temporal action detection (Sinha et al., 10 Jan 2025), robust single-object tracking (Xie et al., 2024), remote physiology (Luo et al., 2024).
• Time Series: Multivariate/univariate forecasting in traffic, electricity, weather, financial data (Wang et al., 2024, Cai et al., 2024, Ma et al., 2024).
• Neuroscience/BCI: EEG-based motor imagery decoding (Yang et al., 2024).
• Sequential Decision-Making: Imitation learning (overcoming the Markov assumption), robotics with long-horizon POMDP structure (Zhou et al., 18 May 2025).
• Motion Synthesis: Human and human–human interaction motion generation in generative, conditional, and cross-agent contexts (Nguyen et al., 14 Oct 2025, Wu et al., 3 Jun 2025).

Recent works suggest continued generalization to additional sequence modeling domains, as well as adaptation for edge deployment, cross-modal fusion, and foundation model pretraining (Ma et al., 2024, Gong et al., 14 Jan 2025).

7. Limitations and Open Challenges

While TMamba blocks are empirically effective and highly scalable, the literature identifies certain open issues:

• On extremely high-dimensional multivariate time series (e.g., >500 channels), performance may marginally trail best specialized networks unless equipped with compressed channel-wise attention or analogous adapters (Ma et al., 2024).
• Optimal design for variable/patch/token scan order remains an active area of research, especially with rapidly varying or sparse input topologies (Cai et al., 2024).
• For tasks involving interleaved spatial–temporal dependencies (e.g., skeleton-based motion), additional multi-branch or alternating spatial–temporal Mamba blocks are necessary for full effectiveness (Wu et al., 3 Jun 2025).
• While linear complexity enables scaling, careful tuning of state-dimension, gating, dropout, and fusion mechanisms is required to balance underfitting and overfitting on domain-specific tasks.

In sum, TMamba constitutes a principled, empirically validated approach to efficient, robust temporal modeling, providing both a theoretical and practical alternative to conventional Transformer or CNN temporal encoders in long-range sequence processing across modalities (Luo et al., 2024, Gong et al., 14 Jan 2025, Wu et al., 2024, Nguyen et al., 14 Oct 2025, Ma et al., 2024).