EEG-CSANet: Multiscale EEG Feature Fusion

• The paper demonstrates that EEG-CSANet’s fusion of multiscale features via centralized sparse attention achieves state-of-the-art decoding performance across various EEG benchmarks.
• It employs a four-branch depth-wise separable convolution structure coupled with multiscale attention and temporal convolutional networks to effectively capture spatial and temporal EEG patterns.
• Empirical results reveal significant gains in accuracy and robustness over previous methods, with reduced computational load enabling practical real-time BCI applications.

Fusion of Multiscale Features via Centralized Sparse-attention Network (EEG-CSANet) is a neural network architecture for spatiotemporal electroencephalography (EEG) signal decoding that integrates multiscale feature extraction, centralized sparse attention-based fusion, and temporal sequence modeling. EEG-CSANet targets the inherent scale diversity and spatial-temporal nonstationarity of brain signals by combining scale-specific convolutional branches with a main-auxiliary attention-driven fusion regime. It has demonstrated state-of-the-art (SOTA) performance across canonical motor imagery, emotion recognition, and vigilance estimation EEG benchmarks (Cai et al., 21 Dec 2025).

1. Network Architecture and Design Rationale

EEG-CSANet employs a depth-wise separable convolutional backbone partitioned into four parallel branches, each dedicated to a distinct temporal scale. The architectural pipeline comprises:

• Data Augmentation (S{paper_content}R): Each EEG trial is segmented into eight blocks, randomly shuffled and recombined within-class, then concatenated with unaugmented data.
• Multi-Branch Temporal + Spatial Convolution: Four branches with 1D temporal kernel sizes $K_i$ in {64, 32, 16, 8}, followed by depth-wise separable spatial convolution (DW-Spa-Conv), extract frequency- and topology-specific features for each scale, yielding feature maps $Z_i \in \mathbb{R}^{B \times U_i \times T_0}$.
• Feature Fusion via Attention:
  • The main branch (largest kernel, slowest rhythm) employs a Multiscale Multi-Head Self-Attention (MSA) block.
  • Each auxiliary branch interfaces with the main via a Multiscale Sparse Cross-Attention (MSCA) block, where feature maps are mutually refined.
  • Each attention block adds a residual path: $M_i = Z_i + \mathrm{MHA}_i$.
• Temporal Convolutional Network (TCN) Head: Each branch’s output undergoes an identical two-layer, dilated TCN and concatenation before classification.

The motivation is to enable simultaneous learning of scale-specific spatial-spectral patterns and their cross-scale interactions while maintaining computational efficiency and semantically guided fusion (Cai et al., 21 Dec 2025).

2. Mathematical Formulations and Attention Mechanisms

The main and auxiliary branches leverage different attention paradigms:

• Multiscale Multi-Head Self-Attention (MSA): For the main branch, three average poolings (kernels {3,5,7}) are summed to produce the input $X$, projected into queries, keys, and values (Q, K, V). For each head,

$A^{(h)} = \frac{Q^{(h)}(K^{(h)})^T}{\sqrt{d_k}}$

Softmax is applied to $A^{(h)}$, and heads are concatenated.

• Multiscale Sparse Cross-Attention (MSCA): For each auxiliary branch, queries derive from the main branch and keys/values from the auxiliary. Top-k sparsification is applied per row—only top-$k_1$ and top-$k_2$ entries are retained, blended via learnable weights ($\alpha, \beta$):

$$\mathrm{Attention} = \alpha\,A'_1\,V + \beta\,A'_2\,V$$

This enforces that only the most semantically relevant cross-scale interactions are preserved, reducing spurious correlation propagation and computational complexity.

Each branch’s resulting representation $M_i$ passes through dilated TCN layers before concatenation and classification (Cai et al., 21 Dec 2025).

3. Spatial and Temporal Feature Extraction Modules

Each convolutional branch executes the following sequence:

• Temporal Conv2D: 1D convolutions along time ($K_i \times 1$), filter count $F_i=16$.
• Depth-wise Separable Spatial Conv: Depthwise spatial kernel ($C \times 1$), depth-multiplier $D = 2$, pointwise 1×1 convolution with $F_6 = 32$ channels.
• Activation and Regularization: Each convolution is followed by BatchNorm, ELU nonlinearity, and 0.5 dropout.
• Average Pooling: Successive poolings ($(8,1), (7,1)$) compress time from $T$ to $T_0 = T/(8 \cdot 7)$.

This schema ensures each branch maps raw EEG sub-bands into spatially resolved, scale-aware feature maps amenable for downstream attention-based fusion (Cai et al., 21 Dec 2025).

4. Hyperparameters, Training Regimes, and Dataset Characteristics

Key settings include:

• Architecture: Four branches; temporal convolution kernels {64,32,16,8}; filters {16,16,16,16}; attention heads $h=8$; pooling sizes {3,5,7}; Top-$k$ per attention row (ratios 2, 3).
• Regularization: 0.5 dropout (convs), 0.3 (TCN), skip connections, data augmentation.
• Training: Adam optimizer, learning rate 0.0009, cross-entropy loss, fixed seed.
• Dataset protocols:
  • BCIC-IV-2A/B: 4 s trials, 22/3 channels, subject-wise splits.
  • HGD: 44 channels, 4 s trials, ∼880 training, ∼160 test per subject.
  • SEED/SEED-VIG: 62/17 channels, 1 s/8 s windows, 15/23 subjects, five-fold cross-validation.

All experiments are conducted in PyTorch on an RTX 2080Ti GPU (Cai et al., 21 Dec 2025).

5. Empirical Results and Comparative Analysis

EEG-CSANet establishes new SOTA across five public EEG benchmarks:

Dataset	Accuracy (%)	Cohen’s κ	Previous Best	Δ (CSANet–prev)
BCIC-IV-2A	88.54 ±8.41	0.8472	85.03	+3.51
BCIC-IV-2B	91.09 ±8.48	0.8218	89.70	+1.39
HGD	96.43 ±4.52	0.9542	95.90	+0.53
SEED	96.03	0.9404	95.70	+0.33
SEED-VIG	90.56	0.7327	90.14	+0.42

Statistical significance is achieved versus all major baselines (paired t-tests, p<0.05 or p<0.01). EEG-CSANet achieves robust generalization across subject variability and task domains without post-hoc parameter tuning (Cai et al., 21 Dec 2025).

6. Ablation Studies and Interpretability

Systematic ablations dissect EEG-CSANet’s components:

• Data Augmentation: Removal induces 7.19% drop (BCIC-2A), demonstrating the importance of S{paper_content}R. Minor effects in SEED datasets are observed.
• Residual Connections: Eliminating these causes the single largest performance decline, affirming their criticality for preserving temporal context.
• Top-k Sparsification / Multiscale Pooling: Removing either in MSCA reduces accuracy, confirming the necessity of both multi-scale and selective attention mechanisms.

Interpretability analyses include:

• UMAP Feature Visualization: Post-training embeddings reveal tight clustering by class.
• Confusion Matrices: Minor errors in confounding class pairs; no class bias.
• Branch-wise Frequency Selectivity: Each temporal branch enhances distinct EEG spectral bands (e.g., kernel 64 amplifies θ/α/β, kernel 8 targets β→γ).

Collectively, these experiments validate both the architectural and physiological sensibility of the multi-branch design (Cai et al., 21 Dec 2025).

7. Computational Complexity and Practical Implications

Parameter count is estimated at 60–80 K, with principal contributions from attention, TCN, and convolutional blocks. Theoretical complexity per batch is dominated by attention ($\mathcal{O}(T_0^2 U)$), though Top-k sparsification ameliorates inference time. Empirical forward pass time on an RTX 2080Ti is 5–15 ms per trial, compatible with real-time brain-computer interface (BCI) settings (Cai et al., 21 Dec 2025).

A plausible implication is that EEG-CSANet’s computational efficiency facilitates deployment in closed-loop BCI or ubiquitous EEG analytics scenarios, despite the scale of attention operations.

---

References:

• "Fusion of Multiscale Features Via Centralized Sparse-attention Network for EEG Decoding" (Cai et al., 21 Dec 2025)
• "CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding" (Zhou et al., 29 Jun 2025)