- The paper demonstrates that fusing expert linguistic features with learned embeddings in tabular models significantly outperforms zero-shot LLMs in multilingual cognitive impairment detection.
- It rigorously evaluates performance across English, Slovene, and Korean datasets using leave-one-out and few-shot protocols to determine optimal modality integration strategies.
- The study underscores the importance of transparent feature engineering and classical ML methods for scalable and interpretable CI diagnostics in clinical settings.
Multilingual Cognitive Impairment Detection with Foundation Models
Scientific Context and Motivation
Cognitive impairment (CI), including mild cognitive impairment (MCI) and Alzheimer’s disease (AD), has profound clinical and societal impact, demanding scalable and accurate diagnostic methodologies. Traditional neuropsychological instruments, though established, are limited by resource-intensiveness, infrequent assessment, and vulnerability to linguistic and cultural confounders. The field has progressively transitioned from classical ML pipelines based on hand-engineered linguistic features toward deployment of large pretrained models, including multilingual LLMs and tabular foundation models. However, systematic evaluation of these approaches—particularly in multilingual, small-data settings—remains sparse.
This study addresses four critical questions: the efficacy of zero-shot LLM-based CI classification, sensitivity to input representation modalities, performance contribution of expert-designed symbolic features in tabular models, and the optimal integration strategies (fusion) of feature and embedding modalities. Experiments are conducted across English, Slovene, and Korean, covering AD and MCI datasets, with rigorous protocols (leave-one-out, few-shot, zero-shot) for direct comparability.
Methodological Overview
Datasets and Feature Engineering
Three datasets are leveraged: English (AD vs. healthy controls, Pitt Corpus), Slovene (AD vs. controls, CogLiTreat project), and Korean (MCI vs. controls, Kang corpus), totaling 260 participants. Inputs are speech transcripts from picture description tasks (English, Slovene) and structured interviews (Korean). Eleven linguistic features—identified as AD-relevant in systematic review literature—are extracted per participant, including speech rate, type-token ratio, repetitiveness, coherence, lexical familiarity, idea density, syntactic complexity, and ratios for verbs, nouns, pronouns.
Embeddings are generated using a frozen multilingual model (embedding-gemma-300m), and fusion modalities (early and late) are constructed to combine symbolic features and embeddings. Preprocessing avoids data leakage via fold-dependent imputation and normalization; fusion compensates for dimensionality imbalance.
Model Families and Evaluation Protocols
Tabular models include TabPFN (in-context tabular foundation model), RealMLP, logistic regression (LR), random forest (RF), SVM (linear/RBF), LightGBM (LGBM), and k-NN. LLM classifiers (gpt-oss-20b, med-gemma-27b) are evaluated under transcript-only, linguistic-features-only, and full-data prompt variants. Leave-one-out (LOO) CV forms the main protocol; few-shot (k=1,2,3,5) episodic sampling is used for small-label settings; LLM evaluation is strictly zero-shot.
Empirical Findings
Zero-Shot LLM Classification and Modalities
Zero-shot LLMs serve as practical no-training baselines (Macro-F1 up to 0.621 for Korean). However, supervised tabular models (particularly those incorporating symbolic features and fusion) significantly exceed LLM performance by 0.18–0.26 Macro-F1 points, even with modest dataset sizes. LLM performance is highly sensitive to input modality: GPT-OSS-20b is maximized with transcript-only in English and Korean, while MedGemma-27B’s domain specialization provides no consistent advantage and often underperforms in linguistic-only settings.
Role of Symbolic Features and Fusion Strategies
Expert-engineered linguistic features provide substantial signal, especially in English and Korean. For English, TabPFN with features achieves 0.814 Macro-F1, outperforming embeddings-only (0.343). Fusion strategies are dataset-dependent—early fusion is optimal for Slovene and competitive in Korean, while features-only dominates in English. Alignment analysis (CKA, Procrustes, Spearman’s ρ, neighborhood overlap/purity metrics) reveals near-orthogonality between features and embeddings for English and Korean, validating fusion strategies; Slovene’s higher alignment suggests embeddings subsume feature information or reflect dataset confounders.
Few-Shot Tabular Models vs. Zero-Shot LLMs
Embedding-based few-shot models approach or surpass zero-shot LLMs in Slovene (Macro-F1 0.846–0.852 at k=1), and LR achieves 0.667 in Korean at k=5, outperforming the best LLM baseline. In English, features-only tabular models remain necessary to surpass LLMs, indicating that labelled data with rich representations is critical when supervision is severely limited.
Practical and Theoretical Implications
The study robustly demonstrates that in small-data, multilingual CI detection, transparent symbolic linguistic features—integrated with strong tabular models and fusion-based strategies—yield reliability and interpretability superior to zero-shot LLM inference. Classical ML models, such as LR and RF, retain competitiveness, echoing findings in recent foundation model literature [hollmann2025accurate, ding2026longitudinalprogressionpredictionalzheimers]. Although tabular foundation models (e.g., TabPFN) show impressive performance on low-resource tasks (2604.06758), their marginal gains over well-tuned classical baselines are dataset-contingent.
For clinical deployment, the interpretability and feature transparency of tabular approaches are crucial, offering explainability advantages over LLMs—whose outputs are influenced by prompt sensitivity and black-box integration of modalities. LLMs, however, provide value as no-training benchmarks and may underpin new hybrid systems via in-context learning or prompt design.
Extending such frameworks to other elicitation paradigms (spontaneous speech, narrative recall), handling missing features robustly, and developing multilingual symbolic features merit further exploration. Analytical alignment between feature and embedding spaces can inform future multimodal fusion architectures. Additional methodological rigor must address dataset confounds, demographic/educational bias, and task generalization for broader validity.
Conclusion
Cross-linguistic CI detection from speech is best addressed by fusing engineered linguistic features with learned embeddings in tabular models, achieving higher accuracy and interpretability than zero-shot LLMs in three languages. The results underscore the enduring utility of classical ML and careful feature engineering in the era of foundation models, while suggesting that prompt-based LLM classifiers are best employed as complementary baselines in small-data settings. For practical diagnostic use, transparent, fusion-based tabular models are preferable, though LLMs may support rapid prototype deployment where labels are unavailable or in resource-scarce languages.