Fusing Pretrained ViTs with TCNet for Enhanced EEG Regression
Abstract: The task of Electroencephalogram (EEG) analysis is paramount to the development of Brain-Computer Interfaces (BCIs). However, to reach the goal of developing robust, useful BCIs depends heavily on the speed and the accuracy at which BCIs can understand neural dynamics. In response to that goal, this paper details the integration of pre-trained Vision Transformers (ViTs) with Temporal Convolutional Networks (TCNet) to enhance the precision of EEG regression. The core of this approach lies in harnessing the sequential data processing strengths of ViTs along with the superior feature extraction capabilities of TCNet, to significantly improve EEG analysis accuracy. In addition, we analyze the importance of how to construct optimal patches for the attention mechanism to analyze, balancing both speed and accuracy tradeoffs. Our results showcase a substantial improvement in regression accuracy, as evidenced by the reduction of Root Mean Square Error (RMSE) from 55.4 to 51.8 on EEGEyeNet's Absolute Position Task, outperforming existing state-of-the-art models. Without sacrificing performance, we increase the speed of this model by an order of magnitude (up to 4.32x faster). This breakthrough not only sets a new benchmark in EEG regression analysis but also opens new avenues for future research in the integration of transformer architectures with specialized feature extraction methods for diverse EEG datasets.
- Physics-informed attention temporal convolutional network for eeg-based motor imagery classification. IEEE Transactions on Industrial Informatics, 19(2):2249–2258, 2022.
- An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
- Jia Deng et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. IEEE, 2009.
- Alexey Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
- Ms-tcn: Multi-stage temporal convolutional network for action segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019.
- Kai Han et al. A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45(1):87–110, 2022.
- Pradeep Hewage et al. Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, 24:16453–16482, 2020.
- Thorir Mar Ingolfsson et al. Eeg-tcnet: An accurate temporal convolutional network for embedded motor-imagery brain–machine interfaces. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2020.
- Ard Kastrati et al. Eegeyenet: a simultaneous electroencephalography and eye-tracking dataset and benchmark for eye movement prediction. arXiv preprint arXiv:2111.05100, 2021.
- Jiyao Liu et al. Spatial-temporal transformers for eeg emotion recognition. In Proceedings of the 6th International Conference on Advances in Artificial Intelligence, 2022.
- David Sabbagh et al. Predictive regression modeling with meg/eeg: from source power to signals and cognitive states. NeuroImage, 222:116893, 2020.
- Evaluating fmri-based estimation of eye gaze during naturalistic viewing. Cerebral Cortex, 30(3):1171–1184, 2020.
- Classification of eeg signals using neural network and logistic regression. Computer methods and programs in biomedicine, 78(2):87–99, 2005.
- Michal Teplan. Fundamentals of eeg measurement. In Measurement Science Review, volume 2, 2002.
- Ashish Vaswani et al. Attention is all you need. In Advances in neural information processing systems, volume 30, 2017.
- Bichen Wu et al. Visual transformers: Token-based image representation and processing for computer vision. arXiv preprint arXiv:2006.03677, 2020.
- Vit2eeg: Leveraging hybrid pretrained vision transformers for eeg data. arXiv preprint arXiv:2308.00454, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.