Mitigating Scoring Errors and Compensating for Nonverbal Subtests in Speech-Based Dementia Assessment

Abstract: Early detection of cognitive impairment relies on neuropsychological tests to minimize subjectivity by assessing multiple cognitive domains. Speech-based evaluation can support diagnostics and improve accessibility, but transcription errors and the omission of nonverbal subtests (e.g., motor skills) limit accuracy. Beyond conventional test scores, speech-derived features can provide additional insights into cognitive status. This study investigates the speech-based evaluation of the German "Syndrom-Kurz-Test," a standardized dementia screening test comprising verbal and motor subtests. We train models that integrate transcript-derived scores and Whisper embeddings per verbal subtest to reduce scoring errors. To compensate for missing motor subtests, we then leverage these fused representations to approximate expert overall ratings. Despite omitting subtests, our models strongly correlate with expert ratings and efficiently and accurately discriminate between cognitive status groups.

• The paper presents a hybrid approach combining rule-based scoring with deep correction to address ASR transcription errors in dementia assessments.
• The study demonstrates that leveraging ASR embeddings in correction and compensation models yields Pearson correlations up to 0.98 and overall score improvements.
• The research shows that speech-only tests, enhanced by deep models, can approximate full clinical scores while maintaining diagnostic discrimination among cognitive states.

Automated Speech-Only Assessment of the SKT for Dementia Screening

Introduction and Motivation

Automated dementia screening based on speech offers the potential for greater accessibility, reduced costs, and scalable monitoring of cognitive decline. However, automating the clinical evaluation of standardized neuropsychological batteries—such as the German Syndrom-Kurz-Test (SKT)—faces two principal challenges: high susceptibility of pathological or dialectal speech to ASR transcription errors, and the inaccessibility of motor (nonverbal) test components via speech-only methods. The paper "Mitigating Scoring Errors and Compensating for Nonverbal Subtests in Speech-Based Dementia Assessment" (2606.18979) addresses both issues by leveraging automatic speech recognition (ASR) with OpenAI Whisper and combining rule-based score extraction with deep neural corrections and compensation strategies.

Figure 1: The end-to-end architecture for automated speech-based dementia assessment utilizing SKT verbal subtests, ASR transcription (Whisper), embedding extraction, deep correction, and motor subtest compensation.

Dataset and Task Scope

The dataset consists of 158 German-speaking subjects (aged 49–89, balanced across no cognitive impairment (NCI), mild cognitive impairment (MCI), and dementia (DEM)), drawn from routine clinical practice settings. Speech recordings cover SKT subtests targeting attention (naming, reading numbers, counting, interference) and memory (immediate recall, delayed recall, recognition). Manual expert scoring followed SKT guidelines, resulting in normed aggregate total scores on a six-point ordinal scale.

Crucially, nonverbal/motor subtests were excluded from the automated pipeline due to their inherent modality mismatch (lack of speech content).

ASR and Feature Representation

The study employs Whisper-small and Whisper-large-v3 models for ASR and embedding extraction. Notably, WER for certain pathology- and task-driven speech (e.g., SKT6 counting, SKT7 interference) exceeds 100%, with ASR hallucination driven by atypical pauses and content sequences. Raw encoder and decoder embeddings are extracted for each utterance, providing high-dimensional latent representations that complement or substitute for error-prone ASR outputs in downstream modeling.

Rule-Based Scoring and Norm Score Construction

Initial subtest scoring is performed with rule-based logic on Whisper transcripts: for memory subtests, object recall and synonym detection is driven by dictionary-augmented string comparison; for attention subtests, temporal alignment to last correct response is timestamped via word-level token mapping. After age- and education-normalization, subtest scores are aggregated per SKT convention.

Deep Correction Models: Mitigation of Transcription Artifacts

To correct error-prone rule-based (RB) subtest scores, a deep correction module is trained for each subtest—accepting as input the RB score and Whisper encoder/decoder embeddings. The architecture fuses RB and embedding streams after separate fully-connected conditioning, feeding a multi-layer perceptron to regress expert-annotated ground truth.

Empirical results with both Whisper-small and -large-v3 show that joint RB+embedding models achieve Pearson correlations with expert scores up to 0.98 (subtest 1, large-v3) and absolute RMSE improvements in ASR-challenging subtests (e.g., SKT6, SKT7). Notably, the correction is most pronounced where WER is highest, exploiting the capacity of embeddings to capture acoustic and paralinguistic features absent or corrupted in the transcript.

Deep Compensation: Approximating Total Scores Without Motor Subtests

Given the speech-only restriction, the next modeling step is to recover total SKT scores—and cognitive status interpretations—despite absent (nonverbal) subtests. Deep compensation models sequentially fuse corrected subtest representations and employ an additional MLP to regress the overall SKT total score from the available subtest set at a given time step $t$.

Addition of each verbal subtest is ordered to maximize the Pearson correlation with expert total SKT scores, stopping once a confidence threshold (e.g., $r_{total}>0.9$) is reached.

Figure 2: Pearson correlation of speech-based SKT total score ($r_{total}$) versus expert ratings as a function of added subtests.

Strong experimental results show that, with only verbal subtests, the deep compensation pipeline achieves $r_{total} = 0.94$ (whisper-small) and $0.95$ (whisper-large-v3), approaching gold-standard performance. Diagnostic discrimination among NCI, MCI, and DEM is maintained, even as half of the clinical battery—specifically, all nonverbal tasks—is omitted.

Subtest Sequencing and Diagnostic Discrimination

Analysis of subtest sequence optimization demonstrates that starting with the interference task (subtest 7), followed by recall (8) and counting (6), produces near-ceiling correlation and accurate group separation, with incremental gains for adding further recall or naming tasks. Clinical workflow constraints (e.g., stimulus dependencies) are respected by aligning subtest order accordingly.

Figure 3: Confusion matrices for deep compensation configurations (whisper-small, RB+ENC deep) for different subtest sequences, stratified by cognitive impairment category (NCI, MCI, DEM).

Discussion: Implications, Limitations, and Future Directions

This work establishes that rule-based ASR scoring, augmented with embedding-based deep correction and compensation, can robustly approximate expert clinical scoring in standardized multi-domain dementia screening—despite severe ASR challenges and the exclusion of non-speech tasks. The system mitigates the impact of variable transcription fidelity in pathological speech, leveraging latent speech features for error correction.

Practical implications include scalable, speech-driven cognitive screening, even in scenarios where full test protocols or multimodal sensor arrays are infeasible. Theoretically, the results support the integration of acoustic-linguistic paralinguistic representations in clinical AI—blurring the distinction between raw ASR accuracy and diagnostically relevant signal.

Challenges persist regarding the robustness of ASR in highly atypical or noisy speech (e.g., heavy dialect, severe disfluency, multi-speaker environments), underlining the need for continued model adaptation, fine-tuning on pathological datasets, and exploration of hybrid or ensemble ASR paradigms. Comparison to alternative clinical assessment batteries and cross-lingual generalization represent open research directions.

Additionally, the deep compensation paradigm offers a template for other neuropsychological and medical assessments whose canonical workflows include multimodal or non-speech elements, motivating expansion into broad clinical AI applications beyond cognitive screening.

Conclusion

The study provides a technically rigorous approach to automated speech-based dementia assessment using the SKT, demonstrating that deep models leveraging ASR embeddings can both mitigate scoring errors and compensate for missing nonverbal subtests. The results underscore the feasibility of accurate, efficient, and accessible cognitive screening from speech alone, suggesting a path toward future AI-driven diagnostic systems with modular modality compensation and robust error mitigation (2606.18979).