Deep learning with convolutional neural networks for EEG decoding and visualization

Abstract: PLEASE READ AND CITE THE REVISED VERSION at Human Brain Mapping: http://onlinelibrary.wiley.com/doi/10.1002/hbm.23730/full Code available here: https://github.com/robintibor/braindecode

• The paper shows that both deep and shallow ConvNets, enhanced by dropout, batch normalization, and ELU, achieve EEG motor decoding accuracies comparable to FBCSP.
• It introduces a cropped training strategy that augments the effective data volume and improves feature learning in the critical 4–f_end Hz frequency band.
• Advanced visualization methods reveal spatial and spectral EEG features, offering actionable insights for refined brain mapping and BCI applications.

Analysis of Convolutional Neural Networks for EEG Decoding and Brain Mapping

The paper presents an extensive exploration of convolutional neural networks (ConvNets) for the decoding of movement-related information from electroencephalography (EEG) data. The paper evaluates multiple ConvNet architectures and designs in comparison to a well-established EEG decoding method, the filter bank common spatial patterns (FBCSP). This research intricately details various elements that contribute to understanding and enhancing the application of ConvNets in EEG-based brain-computer interfaces (BCIs).

The study demonstrates that deep and shallow ConvNets, with suitable design choices, achieve at least comparable accuracies to FBCSP for the task of motor decoding from EEG signals. The performance of ConvNets is attributed in large part to recent advances in machine learning techniques such as dropout, batch normalization, and exponential linear units (ELUs). These elements significantly enhance the networks' ability to generalize from the training data, particularly in grasping EEG spectral variations without predefined band limitations. Hence, the ConvNets excel, particularly in frequency ranges where traditional band power features may not dominate.

Furthermore, the paper proposes and leverages innovative training strategies, notably the cropped training strategy. This training approach improves the networks' ability to learn features from the data, especially in the 4–$f_{end}$ Hz frequency band where traditional trial-wise methods face limitations due to overfitting. The study provides empirical evidence that cropped training considerably boosts decoding performance by augmenting the effective data volume for the ConvNet, thereby harnessing more robust predictions from EEG signals.

Theoretical implications aside, practical applications of the research are highlighted through the inclusion of advanced visualization techniques. These techniques, namely the input-feature unit-output correlation maps and input-perturbation network-prediction correlation maps, offer insights into the spatial and spectral characteristics of features learned by the networks. The visualization methodologies enable a deeper understanding of which aspects of EEG data are being utilized by ConvNets, paving the way for a complementary brain-mapping approach to identify region-specific, frequency-dependent brain signal modulations.

Despite these favorable findings, the paper cautiously acknowledges that ConvNets have not demonstrated considerable superiority over FBCSP in accuracy, prompting further inquiries into ConvNet feature extraction capabilities and potential unknown features. Significant future developments might include expanding training datasets, refining architectural intricacies, or incorporating other neural network variants like recurrent neural networks to harness temporal dependencies in EEG. Also, domain adaptation methods could be explored to enhance performance consistency across different subjects and sessions, which remains a challenge in EEG-based BCI.

In conclusion, the research detailed in the paper underscores both the potential and the current limitations of deep ConvNets in EEG decoding. While ConvNets do not decisively outperform traditional methods, their adaptability, combined with end-to-end learning capabilities and new visualization tools, offers promising avenues for improving EEG-based brain mapping and BCI technology. The findings hold practical importance for the continued development of neural network applications in neuroscientific inquiry and BCI systems, aligning methodological improvements in deep learning with neuroscience's evolving needs.