TopicDiff: A Topic-enriched Diffusion Approach for Multimodal Conversational Emotion Detection
Abstract: Multimodal Conversational Emotion (MCE) detection, generally spanning across the acoustic, vision and language modalities, has attracted increasing interest in the multimedia community. Previous studies predominantly focus on learning contextual information in conversations with only a few considering the topic information in single language modality, while always neglecting the acoustic and vision topic information. On this basis, we propose a model-agnostic Topic-enriched Diffusion (TopicDiff) approach for capturing multimodal topic information in MCE tasks. Particularly, we integrate the diffusion model into neural topic model to alleviate the diversity deficiency problem of neural topic model in capturing topic information. Detailed evaluations demonstrate the significant improvements of TopicDiff over the state-of-the-art MCE baselines, justifying the importance of multimodal topic information to MCE and the effectiveness of TopicDiff in capturing such information. Furthermore, we observe an interesting finding that the topic information in acoustic and vision is more discriminative and robust compared to the language.
- Multimodal topic-enriched auxiliary learning for depression detection. In Proceedings of COLING 2020.
- Label-efficient semantic segmentation with diffusion models. In Proceedings of ICLR 2022.
- IEMOCAP: interactive emotional dyadic motion capture database. Lang. Resour. Evaluation, 42(4):335–359.
- Observing dialogue in therapy: Categorizing and forecasting behavioral codes. In Proceedings of ACL 2019, pages 5599–5611.
- A discrete CVAE for response generation on short-text conversation. In Proceedings of EMNLP 2019, pages 1898–1908.
- COSMIC: commonsense knowledge for emotion identification in conversations. In Findings of EMNLP 2020.
- Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation. In Proceedings of EMNLP 2019.
- Neural topic model with reinforcement learning. In Proceedings of EMNLP 2019.
- ICON: interactive conversational memory network for multimodal emotion detection. In Proceedings of EMNLP 2018.
- Conversational memory network for emotion recognition in dyadic dialogue videos. In Proceedings of NAACL 2018.
- Masked autoencoders are scalable vision learners. In Proceedings of CVPR 2022, pages 15979–15988.
- Denoising diffusion probabilistic models. In Proceedings of NeurIPS 2020.
- MM-DFN: multimodal dynamic fusion network for emotion recognition in conversations. In Proceedings of ICASSP 2022, pages 7037–7041.
- Dialoguecrn: Contextual reasoning networks for emotion recognition in conversations. In Proceedings of ACL 2021.
- MMGCN: multimodal fusion via deep graph convolution network for emotion recognition in conversation. In Proceedings of ACL 2021.
- Collaborative diffusion for multi-modal face generation and editing. In Proceedings of CVPR 2023.
- Relation-aware graph attention networks with relational position encodings for emotion recognition in conversations. In Proceedings of EMNLP 2020.
- Higru: Hierarchical gated recurrent units for utterance-level emotion recognition. In Proceedings of NAACL 2019, pages 397–406.
- Neural attention-aware hierarchical topic model. In Proceedings of EMNLP 2021.
- COGMEN: contextualized GNN based multimodal emotion recognition. In Proceedings of NAACL 2022, pages 4148–4164.
- Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational bayes. In Proceedings of ICLR 2014.
- Past, present, and future: Conversational emotion recognition through structural modeling of psychological knowledge. In Findings of EMNLP 2021.
- Gcnet: Graph completion network for incomplete multimodal learning in conversation. IEEE Trans. Pattern Anal. Mach. Intell., 45(7):8419–8432.
- Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
- Dialoguernn: An attentive RNN for emotion detection in conversations. In Proceedings of AAAI 2019.
- Neural variational inference for text processing. In Proceedings of ICML 2016, volume 48, pages 1727–1736.
- Context-dependent sentiment analysis in user-generated videos. In Proceedings of ACL 2017.
- MELD: A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of ACL 2019.
- Multimodal topic learning for video recommendation. CoRR, abs/2010.13373.
- High-resolution image synthesis with latent diffusion models. In Proceedings of CVPR 2022, pages 10674–10685.
- Mm-diffusion: Learning multi-modal diffusion models for joint audio and video generation. In Proceedings of CVPR 2023.
- Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition. In Proceedings of AAAI 2021.
- Yang Song and Stefano Ermon. 2019. Generative modeling by estimating gradients of the data distribution. In Proceedings of NeurIPS 2019.
- Score-based generative modeling in latent space. In Proceedings of NeurIPS 2021.
- Sentiment classification in customer service dialogue with topic-aware multi-task learning. In Proceedings of AAAI 2020.
- Neural topic modeling with bidirectional adversarial training. In Proceedings of ACL 2020.
- Diffusion models: A comprehensive survey of methods and applications. CoRR, abs/2209.00796.
- M3ED: multi-modal multi-scene multi-label emotional dialogue database. In Proceedings of ACL 2022.
- Mucdn: Mutual conversational detachment network for emotion recognition in multi-party conversations. In Proceedings of COLING 2022.
- Topic-driven and knowledge-aware transformer for dialogue emotion detection. In Proceedings of ACL 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.