EVOKE: Emotion Enabled Virtual Avatar Mapping Using Optimized Knowledge Distillation
Abstract: As virtual environments continue to advance, the demand for immersive and emotionally engaging experiences has grown. Addressing this demand, we introduce Emotion enabled Virtual avatar mapping using Optimized KnowledgE distillation (EVOKE), a lightweight emotion recognition framework designed for the seamless integration of emotion recognition into 3D avatars within virtual environments. Our approach leverages knowledge distillation involving multi-label classification on the publicly available DEAP dataset, which covers valence, arousal, and dominance as primary emotional classes. Remarkably, our distilled model, a CNN with only two convolutional layers and 18 times fewer parameters than the teacher model, achieves competitive results, boasting an accuracy of 87% while demanding far less computational resources. This equilibrium between performance and deployability positions our framework as an ideal choice for virtual environment systems. Furthermore, the multi-label classification outcomes are utilized to map emotions onto custom-designed 3D avatars.
- Emotion recognition in human-computer interaction. IEEE Signal processing magazine, 18(1):32–80, 2001.
- A systematic review on affective computing: Emotion models, databases, and recent advances. Information Fusion, 83:19–52, 2022.
- Danny Oude Bos et al. Eeg-based emotion recognition. The influence of visual and auditory stimuli, 56(3):1–17, 2006.
- Eeg-based emotion recognition: A state-of-the-art review of current trends and opportunities. Computational intelligence and neuroscience, 2020, 2020.
- Real-time eeg-based emotion recognition and its applications. Transactions on Computational Science XII: Special Issue on Cyberworlds, pages 256–277, 2011.
- Emotional state classification from eeg data using machine learning approach. Neurocomputing, 129:94–106, 2014.
- Emotion recognition with machine learning using eeg signals. In 2018 25th national and 3rd international iranian conference on biomedical engineering (ICBME), pages 1–5. IEEE, 2018.
- Seda: Self-ensembling vit with defensive distillation and adversarial training for robust chest x-rays classification, 2023.
- Eeg-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine. In 2009 IEEE international conference on acoustics, speech and signal processing, pages 489–492. IEEE, 2009.
- 4d attention-based neural network for eeg emotion recognition. Cognitive Neurodynamics, pages 1–14, 2022.
- 3dcann: A spatio-temporal convolution attention neural network for eeg emotion recognition. IEEE Journal of Biomedical and Health Informatics, 26(11):5321–5331, 2021.
- Continuous convolutional neural network with 3d input for eeg-based emotion recognition. In Neural Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13–16, 2018, Proceedings, Part VII 25, pages 433–443. Springer, 2018.
- Introducing attention mechanism for eeg signals: Emotion recognition with vision transformers. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 5723–5726. IEEE, 2021.
- Pan-cultural elements in facial displays of emotion. Science, 164(3875):86–88, 1969.
- James A Russell. A circumplex model of affect. Journal of personality and social psychology, 39(6):1161, 1980.
- Evidence for a three-factor theory of emotions. Journal of research in Personality, 11(3):273–294, 1977.
- Albert Mehrabian. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14:261–292, 1996.
- Real-time eeg-based human emotion recognition and visualization. In 2010 international conference on cyberworlds, pages 262–269. IEEE, 2010.
- Deap: A database for emotion analysis; using physiological signals. IEEE transactions on affective computing, 3(1):18–31, 2011.
- Differential entropy feature for eeg-based emotion classification. In 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), pages 81–84. IEEE, 2013.
- Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
- An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations (ICLR), 2021.
- Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
- Tony Mullen. Mastering blender. John Wiley & Sons, 2011.
- Sigmoidf1: A smooth f1 score surrogate loss for multilabel classification. arXiv preprint arXiv:2108.10566, 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.