Time-space-frequency feature Fusion for 3-channel motor imagery classification

Abstract: Low-channel EEG devices are crucial for portable and entertainment applications. However, the low spatial resolution of EEG presents challenges in decoding low-channel motor imagery. This study introduces TSFF-Net, a novel network architecture that integrates time-space-frequency features, effectively compensating for the limitations of single-mode feature extraction networks based on time-series or time-frequency modalities. TSFF-Net comprises four main components: time-frequency representation, time-frequency feature extraction, time-space feature extraction, and feature fusion and classification. Time-frequency representation and feature extraction transform raw EEG signals into time-frequency spectrograms and extract relevant features. The time-space network processes time-series EEG trials as input and extracts temporal-spatial features. Feature fusion employs MMD loss to constrain the distribution of time-frequency and time-space features in the Reproducing Kernel Hilbert Space, subsequently combining these features using a weighted fusion approach to obtain effective time-space-frequency features. Moreover, few studies have explored the decoding of three-channel motor imagery based on time-frequency spectrograms. This study proposes a shallow, lightweight decoding architecture (TSFF-img) based on time-frequency spectrograms and compares its classification performance in low-channel motor imagery with other methods using two publicly available datasets. Experimental results demonstrate that TSFF-Net not only compensates for the shortcomings of single-mode feature extraction networks in EEG decoding, but also outperforms other state-of-the-art methods. Overall, TSFF-Net offers considerable advantages in decoding low-channel motor imagery and provides valuable insights for algorithmically enhancing low-channel EEG decoding.

• The paper introduces TSFF-Net, a network that fuses time, space, and frequency features to enhance MI classification using only 3-channel EEG data.
• It employs continuous wavelet transform for time-frequency representation combined with CNNs for efficient feature extraction.
• The approach outperforms traditional CSP and time-series models, achieving up to 86.4% accuracy and advancing portable BCI technology.

Time-Space-Frequency Feature Fusion Network for Low-Channel Motor Imagery Classification

The work presented in "Time-space-frequency feature Fusion for 3-channel motor imagery classification" introduces a novel network architecture, TSFF-Net, targeting the challenges posed by low-channel electroencephalography (EEG) devices used for motor imagery (MI) applications. Low-channel EEG devices, while essential for portable and entertainment applications, suffer from limitations such as low spatial resolution. This paper proposes a solution that integrates time-space-frequency features to enhance the performance of EEG decoding in scenarios with minimal channels, specifically three channels.

TSFF-Net comprises four integral components: time-frequency representation, time-frequency feature extraction, time-space feature extraction, and a feature fusion and classification module. This study stands out by employing the continuous wavelet transform (CWT) to derive time-frequency representations, thus enabling the conversion of EEG signals into time-frequency spectrograms suitable for further analysis through convolutional neural networks (CNNs). The proposed TSFF-img architecture focuses on light-weighted feature extraction from these spectrograms and demonstrates superior capability to standard image recognition networks like AlexNet, VGG, and ResNet in EEG processing.

Noteworthy numerical findings are reported in this study:

• TSFF-Net achieved significant improvements in classification accuracy over traditional methods, with average accuracies of 85.1% for binary classification tasks (BCI4-2A dataset) and 86.4% (BCI4-2B dataset) using only three EEG channels.
• For quadruple classification tasks, TSFF-Net reached an average accuracy of 65.2% using the same constrained channel setup.

The paper conducted rigorous comparisons with both CSP-based methods and time-series based neural network paradigms, demonstrating that TSFF-Net’s fusion approach significantly outperforms purely time-series or time-frequency approaches. The addition of the Maximum Mean Discrepancy (MMD) loss aids in aligning low-dimensional representation distributions across different features, further enhancing classification performance.

The implications of TSFF-Net extend to both theoretical and practical domains. From a theoretical perspective, the integration of multimodal feature fusion opens avenues for further exploration in low-channel EEG decoding. Methodologically, this study presents a strong case for adopting lightweight spectral analysis in brain-computer interface (BCI) systems, challenging the conventional reliance on complex network architectures.

Future developments in AI, particularly in BCI applications, could look towards enhancing multimodal integration schemes as proposed in TSFF-Net. The promising results achieved in this study suggest that combining complementary views of input data—time, space, and frequency—can mitigate the information loss inherent in low-channel settings. This can potentially catalyze innovations in designing portable and user-friendly EEG devices, ultimately broadening the accessibility and application of BCIs in everyday technology and entertainment.

The study’s methodology, characterized by its attention to computational efficiency and classification precision using limited resources, paves the way for future research in improving EEG usability in low-channel contexts. Such advancements are particularly crucial given the growing interest in wearable and portable EEG systems.