Hungarian Conversational ASR Systems

• Hungarian conversational ASR is a system designed to transcribe informal, multi-speaker Hungarian speech despite challenges like extreme morphological richness and variable acoustics.
• Researchers utilize advanced dataset annotation, subword modeling, and neural language modeling to reduce OOV rates and improve transcription accuracy.
• Innovative techniques such as model distillation, data augmentation, and end-to-end architectures yield significant improvements in word error rates and real-time performance.

Hungarian conversational automatic speech recognition (ASR) systems are engineered for the transcription of informal, spontaneous multi-speaker spoken Hungarian. The field is characterized by linguistic challenges arising from extreme morphological richness and syntactic variability, compounded by the demanding acoustics and disfluencies typical of spontaneous conversation. Recent advances center on dataset creation, subword modeling, neural language modeling, model distillation, augmentation, and simulation strategies specific to Hungarian’s unique properties.

1. Linguistic and Data Challenges in Hungarian Conversational ASR

Hungarian is a morphologically rich, agglutinative language with combinatorial productivity. In moderate-sized corpora, 100,000+ word forms are typical, producing high type/token ratios and a fragmented frequency distribution. Out-of-vocabulary (OOV) rates in conversational speech can reach ≈2.5% at the word level, falling to ≈0.08% when using a 30,000-item morph vocabulary (Tarján et al., 2019). Acoustic data is typically low bandwidth (8 kHz telephony or interviews), informal, and permeated with hesitations, truncated words, slips, and frequent speaker changes (Gedeon et al., 17 Nov 2025). Such conditions elevate ASR perplexity and degrade performance of count-based n-gram models.

The scarcity of publicly available, well-annotated spontaneous and conversational Hungarian corpora has long impeded the field. Recent releases such as BEA-Base (81 h, 140 speakers) (Mihajlik et al., 2022), BEA-Large (255 h, 433 speakers), and BEA-Dialogue (85 h, fully segmented, speaker-disjoint dialogues) address this gap, providing in-domain benchmarks for spontaneous and multi-party ASR and diarization (Gedeon et al., 17 Nov 2025).

2. Dataset Construction and Annotation Paradigms

Initial benchmarks for spontaneous Hungarian ASR relied on the BEA-Base corpus, which includes manually segmented train/dev/eval splits—train-114, dev/eval-repet, dev/eval-spont—with all interviewer and partner audio removed from reference sets to guarantee speaker independence (Mihajlik et al., 2022). BEA-Large substantially expands coverage with 255 h of spontaneous single-speaker data from 433 speakers and augments each segment with demographic and role metadata.

BEA-Dialogue introduces ≈85 h of natural dialogues grouped into ≈30 s segments, using silence-based segmentation and module-level metadata to ensure natural conversational boundaries. Explicit speaker role labels (SPK/EXP/DP) and Organized splits—train, dev, eval—guarantee disjoint speakers across all roles (Gedeon et al., 17 Nov 2025).

Transcription conventions accommodate conversational phenomena: time-aligned utterance transcription at the word level and, for BEA-Dialogue, serialized output training (SOT) with <sc> tokens for speaker changes, so overlapping turns and rapid back-and-forth are accommodated explicitly.

3. Language Modeling and Subword Strategies

Conventional n-gram LMs face critical data sparsity from Hungarian's inflectional explosion. Subword modeling—segmenting text into statistically derived morphs (using Morfessor 2.0, tuned for minimal description length)—reduces vocabulary size from ≈100,000 word types to ≈32,000 morph types, increasing frequency counts per type and dramatically reducing OOV rates (Tarján et al., 2019). Both count-based (BNLM) and neural LLMs (RNNLM, Transformer LM) are retrained using morph-segmented data, achieving improved perplexity and WER.

BNLMs are typically high-order (6-gram cross-sentence, modified Kneser-Ney smoothing, no count threshold/pruning) and support single-pass, real-time decoding (Tarján et al., 2019, Tarján et al., 2020). For neural LMs, both LSTM (650-dim, two-layer) and Transformer architectures (e.g., “GPT-2 medium,” 345M parameters, 24 layers, 1024-dim, 16 heads) are used. Morph-based LMs consistently outperform word-based variants; for example, a morph-based BNLM achieves WER=28.7% versus 29.2% for words (Tarján et al., 2019).

4. Distillation, Augmentation, and Data Simulation

Neural LMs—despite offering lower perplexity (e.g., LSTM RNNLM, PPL=44.6 vs. BNLM 85.7 (Tarján et al., 2019))—incur prohibitive run-time overhead for first-pass decoding. Distillation and augmentation approaches circumvent this:

• RNN-BNLM: The RNNLM generates synthetic text (100M–1B tokens sampling from $P_R$), from which a back-off n-gram model is estimated. The resulting RNN-BNLM can be used in real-time first-pass decoding. Interpolation with the baseline BNLM yields substantial perplexity and WER reduction (e.g., morph-based: interpolated RNN-BNLM recovers ≈40% of RNNLM’s perplexity improvement and achieves up to 8% relative WER reduction vs. the morph BNLM baseline) (Tarján et al., 2019).
• Transformer-based augmentation: A GPT-2 Transformer is pretrained on non-conversational Hungarian (parliament transcripts), then fine-tuned on in-domain conversation. 1B tokens are generated by prefix sampling and randomizing temperature, after which BNLMs are estimated on synthetic data and interpolated with in-domain BNLMs. Subword-based retokenization (Morfessor/BPE, 30–50k subwords) is critical to avoid vocabulary explosion and optimize OOV recall (≈25% with >95% precision) (Tarján et al., 2020).
• Speaker-Aware Simulated Conversations (SASC/C-SASC): To address the scarcity of annotated dialogue, SASC composes synthetic dialogues by weaving single-speaker utterances into two-speaker conversations using empirically estimated Markovian turn and pause statistics. C-SASC further conditions pause distributions on upcoming utterance duration (using kernel density estimation), better aligning with observed conversational timing (Gedeon et al., 4 Feb 2026).

Table: WER Improvements by Advanced Language Modeling (Hungarian Telephone/Call-Center Data)

Approach	Context	Tokenization	WER (%)
BNLM (word)	4-gram	100k words	29.2
RNNLM (word)	∞-gram	50k words	—
RNN-BNLM (1B synth., morph)	4-gram	30k morphs	28.6
Interp. BNLM+RNN-BNLM (1B, morph)	4-gram	30k morphs	27.3
Transformer Aug. (subword, Morfessor)	4-gram	40k subwords	19.6

Subword-based and neural-augmented LMs yield sharp reductions in OOV and enable practical, memory-efficient online operation (Tarján et al., 2019, Tarján et al., 2020).

5. Acoustic Modeling and ASR Baselines

Initial baselines used hybrid HMM-DNNs (Kaldi, 3 layers × 2500 ReLU, 4907 senones, MFCC+LDA+MLLT front-end, 8 kHz) (Tarján et al., 2019). Recent benchmarks employ end-to-end architectures: Fast Conformer-CTC Large (120M params) using NVIDIA NeMo toolkit, QuartzNet (15×3, 18.9M params) with CTC, and CRDNN+GRU CTC+Attention (Mihajlik et al., 2022, Gedeon et al., 17 Nov 2025). Fine-tuning multilingual self-supervised models (wav2vec2-large, XLS-R-53k/300M) on BEA-Base yields WERs as low as 15.6% on eval-spont, a 45% reduction over TDNN-F baselines (Mihajlik et al., 2022).

ASR on BEA-Large (245 h training): WER=14.18% and CER=4.56% on eval-spont with Fast Conformer (Gedeon et al., 17 Nov 2025). BEA-Dialogue (dialogue, SOT): WER=19.24%, cpWER=19.19%, scAcc=82.15% (correct <sc> prediction ratio). Diarization baselines (pyannote.audio, Sortformer) yield DERs from 13.05% to 18.26%.

6. Methodologies for Online, Real-Time Conversational ASR

For deployability, research emphasizes latency, memory footprint, and single-pass operation. Core principles include:

• Single-pass decoding: All BNLM, augmented n-gram, and interpolated models are used in the first-pass WFST decode (no second-pass rescoring), yielding end-to-end latency ≈250 ms and memory ≤1GB for subword LMs (Tarján et al., 2019, Tarján et al., 2020).
• Memory control: N-gram tables estimated from synthetic text (distilled RNN/Transformer) are pruned (entropy/count) to match target memory usage.
• Segmentation and word reconstruction: Morph boundaries are tagged to permit faithful restoration of original word sequences from decoded subword lattices.
• Decoding pipeline: HCLG WFST composition—H: HMM, C: context-dependency, L: lexicon (subwords to phonemes), G: LM—enables online search with explicit subword handling (Tarján et al., 2020).

7. Empirical Benchmarks, Error Patterns, and Future Directions

Empirical benchmarks establish that spontaneous Hungarian remains 2–4× harder to recognize than read speech (e.g., TDNN-F: 28.41% eval-spont vs. 6.26% on eval-repet (Mihajlik et al., 2022); Fast Conformer: 14.18% vs. 4.8% (Gedeon et al., 17 Nov 2025)). Error analysis identifies disfluencies (hesitations “öö,” truncations), overlapping speech, informal register, and rapid turn-taking as major sources of degradation (Gedeon et al., 17 Nov 2025). SASC and C-SASC data augmentation yields an additional ∼15% relative WER reduction over unsimulated baselines, and conditional pause modeling in C-SASC provides systematic but moderate cpWER/cpCER gains, contingent on strong statistical alignment between synthetic and target dialogue corpora (Gedeon et al., 4 Feb 2026). Excessive augmentation (dialogues per speaker >4) and noisy RIR simulation can degrade performance, highlighting the need for careful parameter tuning.

Future research directions include further integration of large pre-trained LMs, domain-adaptation to account for dialect and code-switching, improved multi-party diarization, semi-supervised learning, and richer annotation of conversational phenomena. Datasets such as BEA-Large and BEA-Dialogue, together with published NeMo/SpeechBrain recipes and conditional simulation frameworks, provide blueprints for advancing conversational ASR in Hungarian and analogous morphologically rich, lower-resource languages (Gedeon et al., 17 Nov 2025, Gedeon et al., 4 Feb 2026).