Validating Computational Markers of Depressive Behavior: Cross-Linguistic Speech-Based Depression Detection with Neurophysiological Validation

Abstract: Speech-based depression detection has shown promise as an objective diagnostic tool, yet the cross-linguistic robustness of acoustic markers and their neurobiological underpinnings remain underexplored. This study extends Cross-Data Multilevel Attention (CDMA) framework, initially validated on Italian, to investigate these dimensions using a Chinese Mandarin dataset with Electroencephalography (EEG) recordings. We systematically fuse read speech with spontaneous speech across different emotional valences (positive, neutral, negative) to investigate whether emotional arousal is a more critical factor than valence polarity in enhancing detection performance in speech. Additionally, we establish the first neurophysiological validation for a speech-based depression model by correlating its predictions with neural oscillatory patterns during emotional face processing. Our results demonstrate strong cross-linguistic generalizability of the CDMA framework, achieving state-of-the-art performance (F1-score up to 89.6%) on the Chinese dataset, which is comparable to the previous Italian validation. Critically, emotionally valenced speech (both positive and negative) significantly outperformed neutral speech. This comparable performance between positive and negative tasks supports the emotional arousal hypothesis. Most importantly, EEG analysis revealed significant correlations between the model's speech-derived depression estimates and neural oscillatory patterns (theta and alpha bands), demonstrating alignment with established neural markers of emotional dysregulation in depression. This alignment, combined with the model's cross-linguistic robustness, not only supports that the CDMA framework's approach is a universally applicable and neurobiologically validated strategy but also establishes a novel paradigm for the neurophysiological validation of computational mental health models.

• The paper introduces a novel CDMA framework that fuses read and spontaneous speech with EEG data for robust depression detection.
• It demonstrates that combining emotional arousal with acoustic markers achieves up to an 89.6% F1-score across different languages.
• EEG analysis reveals that variations in theta and alpha bands correlate with depression probabilities, reinforcing the neurophysiological basis of the method.

Validating Computational Markers of Depressive Behavior: Cross-Linguistic Speech-Based Depression Detection with Neurophysiological Validation

Introduction

This study introduces an innovative approach to speech-based depression detection, focusing on its cross-linguistic robustness and neurophysiological validation. The authors extend the Cross-Data Multilevel Attention (CDMA) framework, originally validated on Italian speech, to the Chinese Mandarin dataset, integrating Electroencephalography (EEG) data. The research aims to demonstrate the universality and biological grounding of acoustic markers in depression detection, evaluating the role of emotional arousal versus valence in predictive accuracy.

Methodology

The methodology involves fusing read speech with spontaneous speech of various emotional valences (positive, neutral, negative) to assess emotional arousal's impact on detection performance. EEG recordings were used to correlate the model's predictions with neural oscillatory patterns during emotional face processing, providing a novel validation approach for computational mental health models.

The CDMA framework operates through a two-tiered attention mechanism: Intra-Type Multi-Local Attention (IT-MLA), focusing on distinguishing acoustic patterns within each speech type, and Cross-Type Global Attention (CT-GA), leveraging inter-speech-type information to emphasize shared depression indicators. The final speaker-level prediction combines read and spontaneous speech through majority voting across emotional categories.

Figure 1: The figure illustrates the proposed framework with different colored arrows indicating various speech processing pathways and an icon for majority voting for depression prediction.

Experimental Results

Emotional Arousal Hypothesis

The experimental data demonstrate that the emotional arousal hypothesis holds, with positive and negative spontaneous speech significantly outperforming neutral speech, suggesting that arousal intensity is crucial for prediction accuracy rather than emotional valence. The reading combined with spontaneous speech tasks achieved an F1-score of up to 89.6%, indicating a robust framework for depression detection.

Cross-Linguistic Performance

The cross-linguistic generalization of the CDMA framework was evident as it achieved comparable detection performance across linguistically distinct datasets. This highlights the model's ability to capture universal depression-related acoustic patterns, supporting its applicability in diverse language contexts without requiring extensive modifications or data transformations.

EEG Correlation Insights

The EEG analysis disclosed significant correlations between model-derived depression probabilities and EEG oscillatory activity in theta and alpha bands, suggesting neurophysiological markers' alignment with the CDMA framework's speech-based estimates. For instance, reduced frontal and parieto-occipital alpha power during early stages of fear processing correlated negatively with higher depression logits. These findings underline the potential of EEG as a neurobiological validation tool for computational mental health diagnostics.

Figure 2: Time-frequency representations for the fearful facial expression condition showing significant differences in theta and alpha band power between HCs and MDDs.

Implications and Future Directions

The research establishes a significant advancement in speech-based mental health diagnostics, emphasizing emotional arousal and real-time neurological validation. The cross-linguistic generalizability of the CDMA framework suggests its broad applicability, potentially supporting multilingual mental health assessment in global clinical settings.

Future work should focus on expanding the dataset range and developing intricate multimodal frameworks incorporating additional physiological signals, such as heart rate and skin conductance, to enhance the model's holistic understanding of a patient's mental state. Furthermore, exploring causal relationships through longitudinal studies could elucidate speech and neural feature interactions in depression, facilitating deeper insights into the disorder's cognitive and emotional mechanisms.

Conclusion

This study makes significant contributions by validating emotional arousal as a critical factor in depression detection, confirming the CDMA framework's cross-linguistic robustness, and establishing a neurophysiological basis for its predictive capability. These advances offer promising avenues for enhancing the reliability and universality of computational tools in mental health diagnostics.