RDC Representations: Rate, Distortion & Classification

• RDC representations are a unified framework that balances compression rate, signal distortion, and classification accuracy for semantic tasks.
• They employ deep autoencoder architectures with quantization techniques to optimize both image fidelity and classifier performance.
• Universal RDC models enable scalable video coding by tailoring base and enhancement layers to meet both machine and human perceptual needs.

Rate-Distortion-Classification (RDC) Representations refer to a unified information-theoretic and statistical framework for lossy compression that optimizes the trade-off between compression rate, signal distortion, and classification accuracy. Unlike standard rate-distortion theory, which only considers signal fidelity under a rate constraint, RDC explicitly incorporates semantic task performance—typically classification—into the representation and optimization process. RDC models have become central to machine-oriented compression, enabling end-to-end systems that jointly serve human-perceptual and machine-analytic requirements in vision, signal processing, and communications (Zhang, 2024).

1. Mathematical Formulation and Fundamental Properties

The canonical RDC problem seeks, for a random source $X$ and compressed output $\hat{X}$, to minimize the mutual information (or rate) $I(X;\hat{X})$ under measurement distortion $\mathbb E[\Delta(X,\hat{X})] \leq D$ and a classification error or uncertainty constraint $\epsilon(\hat{X}|C_0)\leq E$ for a designated classifier $C_0$. In Lagrangian form, the objective is

$$L_{\mathrm{RDC}} = I(X;\hat{X}) + \lambda_D\,\mathbb E[\Delta(X,\hat{X})] + \lambda_C\,\epsilon(\hat{X}|C_0)$$

with $\lambda_D,\lambda_C\geq0$ controlling trade-offs between rate, distortion, and classification performance (Zhang, 2024). The RDC function

$$R(D,E) = \min_{p_{\hat{X}|X}} I(X;\hat{X}) \quad\text{s.t.}\quad \mathbb E[\Delta(X,\hat{X})] \leq D,\,\epsilon(\hat{X}|C_0)\leq E$$

is jointly convex and monotonic non-increasing in both constraints. This convexity guarantees that trade-off curves in the $(R,D,E)$ space form convex attainable regions, supporting Pareto front analysis for multi-objective optimization.

2. Closed-Form Solutions and Source Models

For Bernoulli sources under Hamming distortion and binary classification, piecewise closed-form RDC functions emerge: $$R(D,E) = \begin{cases} H_b(p) - H_b(D), & 0\leq D \leq D_1(E) \ I_1(D,E), & D_1(E)<D\leq D_2(E) \ 0, & D>D_2(E) \end{cases}$$ where $H_b(\cdot)$ is the binary entropy, and $D_1$, $D_2$, $I_1$ are derived from the classification error constraint (Zhang, 2024). For Gaussian sources and MSE distortion, the RDC boundary is similarly piecewise, switching between distortion- and classification-active regimes. As derived in (Nguyen et al., 12 Apr 2025), the rate penalty for universal encoding is zero for Gaussian sources but can be quantified by information-theoretic bounds and LP relaxations for general sources, yielding negligible performance loss in many practical cases.

Special cases include the semantic source model $X$ (unobservable state) and $Y$ (extrinsic signal), where RDC representations balance fidelity to both $X$ and $Y$ via auxiliary variables and KKT channel solutions (Liu et al., 2021).

3. Deep Learning Architectures and Training Algorithms

RDC representations are realized via deep autoencoders with quantization bottlenecks and multi-term loss functions. A typical implementation on the MNIST dataset includes:

• Encoder: Multi-layer perceptron mapping pixel values to low-dimensional latent codes $z\in\mathbb R^d$.
• Quantization: Uniform scalar quantization of $z$ over $L$ levels, with straight-through or soft quantization during training to preserve gradients.
• Decoder: Transposed convolutional network reconstructing $\hat{X}$ from quantized $z$.
• Loss: Weighted sum

$$L_{\mathrm{RDC}} = \lambda\|X-\hat{X}\|_2^2 + e(\hat{X}),$$

where $e(\hat{X})$ is the negative log-likelihood from a fixed classifier, balancing pixel-level distortion and semantic accuracy (Zhang, 2024).

• Optimization: SGD or Adam over batches, sweeping $\lambda$ to trace the RDC curve.

Architectures generalize to convolutional encoders/decoders, universal representations (shared encoder with multiple decoders), and integration of additional loss terms for perception (e.g., WGAN, total variation) (Nguyen et al., 12 Apr 2025).

4. Universal RDC Representations and Multi-Task Compression

Universal RDC representations enable broad coverage of distortion-classification tradeoffs with a single fixed encoder and multiple lightweight, task-specific decoders. Theoretical results establish that, for Gaussian sources under MSE distortion, representations trained at the highest required rate can exactly match optimal RDC for all operating points via linear decoders, incurring zero rate penalty (Nguyen et al., 12 Apr 2025).

For non-Gaussian sources, the achievable $(D,C)$ region induced by a fixed representation $Z$ can be characterized as

$$(D,C) \in \Omega(p_{Z|X}) \subseteq \left\{D\geq \mathbb E\|X-\tilde{X}\|^2+\inf_{p_U: H(S|U)\leq C} W_2^2(p_{\tilde{X}},p_U)\right\}$$

where $\tilde{X}=\mathbb E[X|Z]$ and $W_2^2$ denotes squared 2-Wasserstein distance (Nguyen et al., 14 Apr 2025). Penalties for universal encoding are quantitatively small (1–2% distortion increase) as verified on MNIST and SVHN datasets.

5. Empirical Analysis and Quantitative Trade-Offs

Experimental evaluation on MNIST reveals:

• For fixed rate (e.g., $L=3$, 4.75 bits/sample), sharper reconstructions and lower classification error are observed as the RDC weight $\lambda$ increases.
• Distortion and classification loss trade off along smooth, convex curves.
• At representative rates:
  • $L=2$ (3 bits): MSE = 0.10–0.12, classification accuracy = 90–92%
  • $L=5$ (6.96 bits): MSE = 0.02–0.03, classification accuracy = 97–98%
• Universal encoders with retrained decoders trace nearly identical curves to individually optimized (end-to-end) RDC models, confirming negligible penalty in practical setups (Zhang, 2024, Nguyen et al., 12 Apr 2025).

6. Implications for Human-Machine Compression and Scalable Video Coding

RDC modeling enables principled design of codecs that serve both human-perceptual fidelity and machine-analytic accuracy. In Video Coding for Machine (VCM), RDC imparts a blueprint for scalable, layered bitstreams:

• Base layer optimized for machine vision tasks (e.g., object detection, classification).
• Enhancement layer tuned for human perceptual quality. Selections on the convex RDC trade-off surface dictate optimal (rate, distortion, classification) parameterization, guiding operating point determination under competing requirements (Zhang, 2024).

Extensions include multi-task RDC for segmentation/detection, advanced entropy models for tight rate control, and adaptation to streaming scenarios, furthering the integration of semantic constraints into compression pipelines.

7. Theoretical Connections and Generalizations

RDC representations generalize classical rate-distortion and Information Bottleneck (IB) frameworks by adding explicit semantic (classification) constraints at the compressed output. They encompass ternary trade-offs (rate, distortion, classification) and, when further extended, combine perceptual quality constraints (RDPC, rate-distortion-perception-classification) relevant to joint human-machine and GAN-driven tasks (Fang et al., 2023, Wang et al., 2024).

RDC theory provides closed-form and operational bounds for both information-theoretic and practical deep-learning codecs, offering unified recipes for structured representation learning, interpretable clustering/classification, and hierarchical coding (Lu et al., 2023).

---

The RDC theory thus supplies a rigorously grounded, experimentally validated framework for designing compressors and representations that natively balance rate efficiency, signal distortion, and semantic task performance in modern visual and signal analysis applications.