Expurgated Exponents in JSCC

• The paper introduces a two-class expurgated coding method that achieves optimal JSCC error exponents with reduced complexity compared to traditional multi-class schemes.
• It employs joint source-channel analysis using type-class methods and minimax optimization to improve error bounds, particularly at low transmission rates.
• The methodology leverages Carathéodory’s theorem to validate that two-class partitioning suffices to match the optimal expurgated performance bounds in JSCC.

Expurgated error exponents in joint source–channel coding (JSCC) quantify the exponential rate at which the maximal or average decoding error probability decays in optimal transmission schemes over discrete memoryless channels (DMC) for discrete memoryless sources (DMS). Expurgated exponents provide sharper bounds than conventional random-coding exponents, particularly at low rates, and are central in understanding the fundamental limits of reliable communication. Recent research demonstrates that these exponents are optimally obtained through a two-class partitioning of source sequences, reducing the complexity of earlier multi-class methods while fully achieving the best known performance bounds.

1. Mathematical Framework for Expurgated JSCC Exponents

In the JSCC setting, a DMS with alphabet $\mathcal{V}$ emits messages $\mathbf{v} \in \mathcal{V}^k$ distributed as $P_V^k(\mathbf{v}) = \prod_{i=1}^k P_V(v_i)$. These messages are mapped by an encoder $f: \mathcal{V}^k \to \mathcal{X}^n$ to channel input sequences for a DMC $W: \mathcal{X} \to \mathcal{Y}$, used $n$ times. At the receiver, a MAP or ML decoder $g: \mathcal{Y}^n \to \mathcal{V}^k$ reconstructs the source sequence. The system operates at a transmission rate $t = k/n$.

The performance metric is the maximal error probability

$$p_e = \max_{\mathbf{v} \in \mathcal{V}^k} \Pr\left[ g(\mathbf{Y}) \neq \mathbf{v} \mid \mathbf{V} = \mathbf{v} \right],$$

and the fundamental object of study is the largest exponent $E$ such that $p_e \leq \exp(-n E + o(n))$ as $n \to \infty$.

A type-class formalism is used: the empirical distribution (type) $\hat{P}_{\mathbf{v}}$ for source sequences and $\hat{P}_{\mathbf{x}}$ for codewords are central for partitioning and codebook construction. Bhattacharyya distance is the fundamental metric for quantifying codeword distinguishability:

$$d_B(x, \bar{x}) = -\log \sum_y \sqrt{W(y|x) W(y|\bar{x})}.$$

The expurgated JSCC exponent is established as a minimax optimization over source reliability and channel expurgated metrics, involving sums over types and relative entropy terms (Moeini et al., 21 Jan 2026, Moeini et al., 4 Jul 2025).

2. Classical and Expurgated Error Exponents

The classical random-coding exponent for JSCC is formulated as

$$E_{J, \rm rc}(t, P_V) = \max_Q \sup_{0 \leq \rho \leq 1} \left\{ E_0(\rho, Q) - \rho R_c \right\} - t E_s(\rho, P_V),$$

where $E_0$ is the channel Gallager function and $E_s$ is the source reliability function. While this exponent characterizes most practical codes, it is known to be suboptimal for sufficiently low transmission rates.

Expurgated exponents seek to improve these bounds by "expurgating" (removing) poor codeword pairs. Csiszár established two expurgated bounds for JSCC—the so-called "type-partitioned" exponent is:

$$E_{J,2}^{\rm ex}(t,P_V) = \min_{R \geq 0} \left\{ t\,e(R/t, P_V) + \max_{Q \in \mathcal{P}(\mathcal{X})} E'_{\rm ex}(Q, R) \right\},$$

where

• $$e(R/t, P_V) = \min_{Q_V: H(Q_V) \geq R/t} D(Q_V \| P_V)$$,
• $E'_{\rm ex}(Q, R)$ is a channel-dependent expurgated function:

$$E'_{\rm ex}(Q, R) = \min_{ \substack{P_{X\bar X}: P_X = P_{\bar X} = Q \ I_P(X;\bar X) \leq R }} \Big( \mathbb{E}_P [d_B(X, \bar{X})] + I_P(X; \bar{X}) - R \Big ).$$

This exponent can be dualized into the Gallager-like supremum form using concave envelopes in the parameter $\rho \geq 1$.

3. Two-Class Expurgated Coding: Construction and Theory

Recent advances show that the optimal expurgated exponent for JSCC can be achieved through a two-class partitioning paradigm, drastically reducing the complexity from the original type-partitioned approach, which required up to $N_k = \operatorname{Poly}(k)$ distinct classes (Moeini et al., 21 Jan 2026, Moeini et al., 4 Jul 2025).

Two-Class Construction:

• Partition source sequences based on type entropy:
  • $$\mathcal{A}_1 = \{\mathbf{v}: H(\hat{P}_{\mathbf{v}}) \leq R_0 / t \}$$
  • $$\mathcal{A}_2 = \{\mathbf{v}: H(\hat{P}_{\mathbf{v}}) > R_0 / t \}$$
  • where $R_0$ is a threshold parameter.
• Assign distinct input distributions $Q_1$ and $Q_2$ for codeword generation:
  • For $\mathbf{v} \in \mathcal{A}_c$, generate codewords of empirical type $Q_c$.
• Decoding proceeds by MAP or ML-type metrics, possibly incorporating source probabilities.

This two-class method achieves the JSCC expurgated exponent via Carathéodory's theorem: the concave hull of $E'_x(Q, \rho)$ is described by two distributions $Q_1$ and $Q_2$, and the threshold $R_0$ precisely partitions the types to maximize the exponent.

4. Mathematical Characterization of the Exponents

The optimized expurgated exponent for the two-class JSCC scheme is:

$$E_{J,{\rm ex}}(t, P_V) = \sup_{\lambda \geq 1} \left\{ \overline{E}_x'(\lambda) - t E_s(\lambda, P_V) \right\}$$

where

• $$E_s(\lambda, P_V) = \log \left( \sum_v P_V(v)^{1/(1+\lambda)} \right)^{1+\lambda}$$,
• $$E_x'(Q, \rho) = -\rho \sum_x Q(x) \log\left( \sum_{\bar{x}} Q(\bar{x}) e^{-d_B(x, \bar{x}) / \rho} \right )$$,
• $\overline{E}_x'(\lambda)$ is the concave hull in $\rho$ of $E_x'(Q, \rho)$, maximized over $Q$.

The minimax form (primal) and its dual (supremum) are precisely linked, and for optimal $Q_1, Q_2$, this construction exactly recovers Csiszár's type-partitioned exponent (Moeini et al., 4 Jul 2025).

Formulas central to construction and analysis include:

E_{J,2}^{\rm ex}(t,P_V) = \min_{R\ge0} \left\{ t e(R/t, P_V) + \max_{Q \in \mathcal{P}(\mathcal{X})} E'_{\rm ex}(Q, R) \right\} E'_{\rm ex}(Q, R) = \min_{P_{X\bar X}: P_X = P_{\bar X} = Q,\, I_P(X;\bar X) \leq R} \left\{ \mathbb{E}_P[ d_B(X, \bar{X}) ] + I_P(X; \bar{X}) - R \right\}

and in Gallager-type dual,

E_x'(Q, \rho) = -\rho \sum_x Q(x) \log \sum_{\bar{x}} Q(\bar{x}) e^{-d_B(x, \bar{x})/\rho}

5. Random Coding, Expurgation, and Achievability

The two-class expurgated ensemble can be analyzed in both i.i.d. and constant-composition settings. Random Gilbert–Varshamov (GV) code ensembles with expurgation extend this framework: source type-classes are divided into a finite number of coding classes, each assigned a distribution $Q_c$. Codewords are chosen subject to minimum mutual information ("distance") constraints. The average error probability is bounded by summing over all source and codeword type pairs, and tightening the bounding constraints recovers either the random-coding or expurgated exponents (Moeini et al., 21 Jan 2026).

The GV ensemble simultaneously attains the maximum of the random-coding and expurgated exponents for all source types, and aligns exactly with the critical-rate behavior familiar from channel coding. The dual attainment shows that the expurgated bound is tight for JSCC. Improvements are conjectured only for non-i.i.d. or sphere-packing codes beyond the critical rate (Moeini et al., 21 Jan 2026).

6. Optimality, Extensions, and Open Problems

The two-class expurgated scheme is optimal in the sense that, for any partitioning with $m \geq 2$ classes, the exponent cannot exceed the concave envelope already achieved with two classes:

• For non-optimal choices of $Q_1$ and $Q_2$, two-class coding may strictly outperform single-class coding.
• For optimal codes, both exponents coincide, and two-class coding never underperforms the single-class optimum (Moeini et al., 21 Jan 2026).
• Open questions remain whether with more general code ensembles, channels with memory, or additional classes, strictly larger exponents can be achieved, but current theory and numerical examples suggest sufficiency of two classes in the DMS–DMC setting (Moeini et al., 4 Jul 2025).

7. Comparison with Classical Bounds and Implications

Expurgation strictly improves on random-coding exponents at low rates (below the critical rate corresponding to the channel random-coding ensemble), a phenomenon that carries directly into JSCC. The two-class scheme both simplifies the implementation and matches the optimal theoretical bounds derived previously for much finer partitions of source types (Moeini et al., 4 Jul 2025). In the pure channel-coding limit ($t \to 0$), the scheme recovers Csiszár and Körner's classical maximal-of-random- and expurgated-exponent results, providing a unified and tight framework for error exponent analysis across information transmission tasks (Moeini et al., 21 Jan 2026).