A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddings

Abstract: Spatial covariance matrices of EEG signals are Symmetric Positive Definite (SPD) and lie on a Riemannian manifold, yet the theoretical connection between embedding geometry and optimization dynamics remains unexplored. We provide a formal analysis linking embedding choice to gradient conditioning and numerical stability for SPD manifolds, establishing three theoretical results: (1) BWSPD's $\sqrtκ$ gradient conditioning (vs $κ$ for Log-Euclidean) via Daleckii-Kreĭn matrices provides better gradient conditioning on high-dimensional inputs ($d \geq 22$), with this advantage reducing on low-dimensional inputs ($d \leq 8$) where eigendecomposition overhead dominates; (2) Embedding-Space Batch Normalization (BN-Embed) approximates Riemannian normalization up to $O(\varepsilon^2)$ error, yielding $+26\%$ accuracy on 56-channel ERP data but negligible effect on 8-channel SSVEP data, matching the channel-count-dependent prediction; (3) bi-Lipschitz bounds prove BWSPD tokens preserve manifold distances with distortion governed solely by the condition ratio $κ$. We validate these predictions via a unified Transformer framework comparing BWSPD, Log-Euclidean, and Euclidean embeddings within identical architecture across 1,500+ runs on three EEG paradigms (motor imagery, ERP, SSVEP; 36 subjects). Our Log-Euclidean Transformer achieves state-of-the-art performance on all datasets, substantially outperforming classical Riemannian classifiers and recent SPD baselines, while BWSPD offers competitive accuracy with similar training time.

• The paper presents a unified SPD Token Transformer that leverages geometric embeddings (BWSPD, Log-Euclidean, Euclidean) to improve EEG classification accuracy and training stability.
• The paper demonstrates that BWSPD embeddings yield superior gradient conditioning and distance preservation, enhancing optimization dynamics in high-dimensional settings.
• The paper validates that multi-band tokenization and BN-Embed significantly boost performance, achieving up to 99.45% accuracy with low computational costs on diverse EEG benchmarks.

Unified SPD Token Transformer for EEG Classification: Geometric Embedding Analysis and Empirical Performance

Introduction

This paper proposes a unified SPD Token Transformer (STT) framework for EEG classification, with a systematic theoretical and empirical comparison of geometric embedding strategies for symmetric positive definite (SPD) covariance matrices on the Riemannian manifold. The study targets three embedding paradigms—Bures-Wasserstein (BWSPD), Log-Euclidean, and Euclidean—analyzing their impact on gradient conditioning, numerical stability, and class separation capacity under deep Transformer architectures. The framework establishes formal connections between embedding geometry, optimization dynamics, and empirical accuracy, leveraging over 1,500 training runs across three diverse EEG paradigms (motor imagery, ERP, SSVEP) and 36 subjects. The analysis includes ablation on batch normalization in embedding space, multi-band tokenization, and computational efficiency, markedly advancing the understanding of geometric deep learning for EEG-based BCIs.

Theoretical Analysis of Geometric Embeddings

The authors derive three core theoretical results:

1. Gradient Conditioning via Daleckii-Krein Framework: They show BWSPD (matrix square-root) embeddings offer VK gradient conditioning, which is quadratically superior to Log-Euclidean’s k conditioning in high-dimensional settings ($d \geq 22$). This theoretically yields better optimization dynamics and numerical stability.
2. Batch Normalization in Embedding Space (BN-Embed): Standard batch normalization in token space is theoretically equivalent to Riemannian normalization up to $O(\epsilon^2)$ error, contingent on within-batch dispersion. This effect is pronounced in high-channel regimes but negligible for modest channel counts.
3. Bi-Lipschitz Distance Preservation: BWSPD tokens rigorously preserve manifold distances, with Euclidean distance in token space tightly controlling distortion by the condition ratio $k$.

Collectively, these results resolve previous ambiguities regarding the relationship between embedding choice and model trainability, convergence, and geometric fidelity on SPD manifolds for EEG signals.

Unified Transformer Framework and Embedding Strategies

The framework unifies processing of three geometric embeddings within identical Transformer blocks, ensuring that observed differences are strictly due to token geometry. Each covariance matrix $\mathbf{C} \in S_d$ is embedded as a token vector:

• BWSPD: $x = \text{triu}(\sqrt{\mathbf{C}})$ via eigendecomposition,
• Log-Euclidean: $x = \text{triu}(\log(\mathbf{C}))$,
• Euclidean: $x = \text{triu}(\mathbf{C})$.

Tokens are projected, position-encoded, and normalized (BN-Embed), then processed via multi-layer Transformer encoders. Multi-band tokenization ($T>1$) enables sequence modeling by embedding covariances from multiple frequency bands as separate tokens.

Empirical Validation and Performance

Dataset Coverage and Experiment Design

The framework is validated on three benchmarks:

• BCI2a: 4-class motor imagery, 22 channels.
• BCIcha: 2-class ERP, 56 channels.
• MAMEM: 5-class SSVEP, 8 channels.

All methods are assessed via cross-subject and per-subject protocols, using Adam optimizer, fixed splits, and paired significance testing.

Main Observations

• Log-Euclidean Embedding: Achieves state-of-the-art accuracy on all datasets—BCI2a: 95.37%, BCIcha: 95.21%, MAMEM: 99.07%. Outperforms classical Riemannian classifiers and SPD deep learning baselines by large margins (up to +70pp on MAMEM).
• BWSPD: Provides competitive accuracy (BCIcha: 90.74%, MAMEM: 81.70%), approximates Log-Euclidean in high-dimensional settings, and offers similar training time (0.28-0.30s/epoch).
• Multi-Band Tokenization: Strongly boosts accuracy and stability (BCI2a: 99.33%±0.39%, +3.96pp, $\sim$96% variance reduction; BCIcha: 99.45%±0.96%, +4.24pp).
• BN-Embed: Critical for high-channel data (BCIcha, +26% accuracy), insignificant for low-channel inputs (MAMEM).
• Computational Efficiency: SPD token approaches have $700\times$ lower FLOPs and minimal GPU memory (BWSPD: 26.14MB) compared to raw time-series deep models.
• Cross-Subject Generalization: Performance degrades markedly due to subject-specific spatial patterns, consistent with prior literature. Alignment techniques such as Euclidean Alignment partially mitigate this issue.

Geometry-Dependent Performance

Results elucidate that embedding selection should be dataset- and task-specific. Log-Euclidean excels for frequency-localized, multi-class problems (e.g., motor imagery). BWSPD is preferable for high-channel, two-class ERP settings where gradient conditioning is paramount. Euclidean baseline generally fails to recover manifold structure unless class separation is trivial.

Implications and Future Directions

Practical Implications

• Principled Embedding Selection: Theoretical analysis guides embedding choice according to dimension, class structure, and numerical stability.
• BN-Embed as Standard Practice: Should be employed for high-dimensional token spaces ($D_{\text{token}} \geq 253$) to ensure scale consistency and training stability.
• Efficient Real-Time BCI Deployment: Low FLOPs and GPU memory footprint suit STT models for edge or wearable BCI applications.
• Multi-Band Tokenization: Provides empirically validated gains for temporal/spectral feature integration.

Limitations

• Cross-subject generalization remains challenging, necessitating further work in Riemannian alignment and domain adaptation methods.
• Embedding performance can vary substantially with dataset and class configuration; thus model selection should be tightly coupled to downstream application requirements.

Speculations for Future Research

• Extend controlled comparison to additional embedding families (Cholesky, affine-invariant).
• Integrate more sophisticated domain adaptation pipelines for robust subject-agnostic BCI decoding.
• Analyze attention patterns in multi-band models to identify neurophysiological correlates of task performance.

Conclusion

This work establishes a rigorous theoretical and empirical foundation for geometric embedding selection in SPD-matrix-based EEG classification via deep Transformers. The Log-Euclidean embedding is empirically optimal across canonical BCI datasets while BWSPD provides competitive alternatives especially in high-dimensional regimes. Multi-band tokenization unambiguously enhances performance and stability. The presented analysis enables precise, dataset-guided architectural and algorithmic decisions for deploying geometric deep learning in brain-computer interfacing. The framework and results offer a substantial resource for future research in geometric deep learning, neuroengineering, and practical BCI deployment.

Reference: "A Unified SPD Token Transformer Framework for EEG Classification: Systematic Comparison of Geometric Embeddings" (2601.21521).