Denoising Diffusion Consistent Models (DDCM)

• DDCM is a family of techniques that enforces consistency in diffusion sampling by incorporating quantized noise injection, guidance, and error-correction steps.
• The framework modifies the reverse diffusion process with a discrete codebook, significantly enhancing zero-shot image compression and robust restoration in inverse problems.
• Incorporating consistency losses mitigates sampling drift, enabling effective multimodal generation and inversion-free editing with improved perceptual metrics.

Denoising Diffusion Consistent Model (DDCM) encompasses a family of techniques for generative modeling, compression, editing, inverse problems, and multi-modal restoration, all unified by the principle of enforcing consistency between intermediate model predictions across the diffusion sampling trajectory. Initially emerging to address limitations of Denoising Diffusion Probabilistic Models (DDPMs) in compression and inverse problems, and later generalized for broader classes of diffusion models, DDCMs and their variants formalize the quantization, guidance, or error-correction steps within the diffusion process to achieve desirable trade-offs between fidelity, perceptual quality, and representation compactness.

1. DDCM: Foundational Concepts and Formulation

The core DDCM framework relies on the standard diffusion family forward marginal:

$$x_t = s(t)\,x_0 + \sigma(t)\,\varepsilon,\qquad \varepsilon \sim \mathcal{N}(0,I)$$

for $t \in [0, T]$, where $x_0$ is the clean datum, and $s(t),\sigma(t)$ are schedule functions (e.g. in DDPM, $s(t)=\sqrt{\alpha_t}$, $\sigma(t)=\sqrt{1-\alpha_t}$). The DDCM construction modifies the reverse process (sampling or denoising) so that the added noise at each step is drawn not from the continuous Gaussian, but from a discrete, pre-defined codebook or is otherwise quantized or guided to remain consistent with model predictions.

For instance, in the discrete-noise DDCM for DDPMs (Kong, 17 Nov 2025), the backward (denoising) step replaces the noise sample $\varepsilon$ with a codevector $e_c\in\mathcal Z=\{e_i\}_{i=1}^K$, selected to minimize the $\ell_2$ distance from the true residual, i.e. $e_c = \arg\min_{e_i} \| e_i - e' \|^2$, with $e'$ the inferred target noise for recovery.

This quantization is extended to various diffusion variants by observing that all such models admit marginals of the above form, and the discrete codebook selection can be cleanly inserted as a plug-in step in generalized samplers, including ODE/consistency-based, flow-matching, and score-based models.

2. DDCM for Compression: Algorithmic Structure and Quantization

The original application of DDCM focused on zero-shot image compression. In this context, the reverse diffusion sampling is modified so that, at each step, the stochastic noise realization is replaced by a codebook vector drawn from a reproducible set (fixed by a pseudo-random seed), yielding a compact and invertible representation:

• Forward step (compression): Standard noisy process, e.g. $x_t = \sqrt{\alpha_t}\,x_0 + \sqrt{1-\alpha_t}\,e$, with $e\in\mathcal Z$.
• Reverse step (reconstruction): For each $t$, the denoiser provides a predicted clean sample $\hat{x}_0$ and residual $r_t = x_0 - \hat{x}_0$; the codevector maximizing correlation with $r_t$ is picked—see

$k_t = \arg\max_{k}\langle z_t^{(k)}, r_t\rangle$

and encoded. The set of codebook indices encodes the latent, and sampling the reverse process with these yields an approximate reconstruction.

• Training objective: The underlying diffusion backbone is pretrained with standard MSE/DDPM loss,

$$\mathcal{L}_{\mathrm{DDPM}} = \mathbb{E}_{x_0,\varepsilon,t} \| \varepsilon - \varepsilon_\theta(x_t,t) \|^2$$

Codebook entries remain fixed and are not further optimized.

• Rate-control and trade-offs: The rate-distortion-perception trade-off is governed by the number of steps $T$, codebook size $K$, and sparsity in multi-atom encoding (e.g. with matching pursuit for multiple atoms per step (Vaisman et al., 9 Nov 2025)).

Turbo-DDCM advances this by providing closed-form sparse quantized least-squares selection over large codebooks, reducing sampling runtime by $40\times$–$50\times$ (Vaisman et al., 9 Nov 2025).

3. Consistency and Generalized DDCM (gDDCM)

The consistency paradigm in diffusion models emerges from the observation that standard denoisers are trained on non-drifted data, while recursive sampling leads to sampling drift, compromising fidelity and introducing artifacts (Daras et al., 2023, Cheng et al., 2024). DDCMs address this by incorporating consistency constraints during training or sampling, ensuring that predictions remain invariant (consistent) as one progresses backward along the learned reverse SDE or ODE.

The learned denoiser or score function $h_\theta(x,t)$ is consistent if, at all times,

$$h_\theta(x, t) = \mathbb{E}_{\text{reverse-SDE}}[x_0\mid x_t = x]$$

This can be regularized via a roll-back consistency loss:

$$L_{\text{cons}}(\theta) = \mathbb{E}_{t,x_t} \| s_\theta(x_t, t) - s_\theta(x_{t-\Delta t}, t-\Delta t)\|^2,$$

where $x_{t-\Delta t}$ is sampled by the learned reverse process.

gDDCM leverages this property to generalize codebook-based compression, quantized-noise injection, or tokenization from DDPMs to any diffusion variant by combining

• a backward ODE step,
• a forward “re-noise” step with codebook quantization of injected noise,
• and the associated index encoding, resulting in effective, model-agnostic latent discretization (Kong, 17 Nov 2025).

4. DDCM for Inverse Problems and Restoration

For inverse problems and restoration, DDCM frameworks center on two main innovations: explicit modeling of the forward (measurement) process, and consistent denoising plus error correction through data-consistent training or explicit residual correction (Fabian et al., 2023, Cheng et al., 2024).

• Forward model: The observation is generated as $$y_t = A_t(x_0) + z_t,\quad z_t \sim \mathcal{N}(0, \sigma_t^2 I)$$, with $A_t$ a deterministic degradation operator and $\sigma_t$ a time-dependent noise scale.
• Reverse process: A network $\Phi_\theta(y_t, t)$ predicts $x_0$; incremental reconstruction is enforced by reconstructing $A_{t-\Delta t}(\Phi_\theta(y_t, t))$ to match the true (noiseless) measurement at earlier steps.
• Data Consistency: Every reverse step (denoising or reconstruction) maintains agreement with the measured data. This is sometimes reinforced by an explicit posterior-score guidance term and can be rigorously shown to ensure measurement consistency throughout the trajectory.
• Early Stopping for Perception–Distortion Trade-off: Sampling is halted at an intermediate reverse time $t_\text{stop}$, enabling control over the perception–distortion balance: PSNR/SSIM peak at intermediate $t$, while perceptual metrics monotonically improve as $t\to 0$.

Empirically, this approach yields better alignment with ground-truth measurements and stronger robustness to cumulative error compared to standard DDM training, with quantitative improvements in PSNR, LPIPS, and FID across a range of restoration tasks (Cheng et al., 2024).

5. DDCM in Multimodal and Coordinated Generation Tasks

DDCM has been adapted for tasks requiring cross-modal consistency, such as HDR image synthesis from multiple LDR exposures. In this regime, multiple denoising processes are run in parallel—one per bracket/exposure—and coupled via an explicit consistency loss (Bemana et al., 2024):

$$L_{\text{total}} = L_{\text{diffusion}} + \lambda_{\text{consistency}} L_{\text{consistency}},$$

where $L_{\text{consistency}}$ comprises pairwise penalties that enforce agreement between re-exposed adjacent brackets, considering quantization and physical exposure constraints.

At sampling time, the consistency loss enters as a posterior energy term in the denoising update, with time-dependent weighting schedules to balance independence in early noisy steps with cross-exposure fusion in later steps.

This framework enables HDR synthesis with state-of-the-art FID, perceptual plausibility, and full-reference HDR metrics, even without retraining on HDR datasets (Bemana et al., 2024).

6. DDCM for Editing and Virtual Inversion

In inversion-free editing, DDCM formalizes a “virtual inversion” technique, enabling text-guided semantic manipulation in diffusion models without explicit inversion optimization (Xu et al., 2023). By selecting a variance schedule ($\sigma_t=\sqrt{1-\alpha_{t-1}}$) that aligns the backward solver with multi-step consistency sampling, it is possible to guarantee exact recovery to the target latent $z_0$:

$$z_{t-1} = \sqrt{\alpha_{t-1}}\,z_0 + \sqrt{1-\alpha_{t-1}}\,\varepsilon_t$$

This enables consistent, efficient, and faithful editing by propagating modifications only insofar as needed for layout changes or prompt conditioning, while preserving background and structure without reconstruction artifacts. Unified attention control further enables simultaneous rigid and non-rigid edits.

In empirical evaluations, DDCM-based inversion-free editing matches heavy inversion baselines (e.g. Prompt-to-Prompt, MasaCtrl) in CLIP-similarity and PSNR at a fraction of the compute cost, and exhibits strong performance in both image-to-image and semantic editing (Xu et al., 2023).

7. Limitations, Practical Considerations, and Future Directions

DDCM methods introduce novel hyperparameters: codebook schedule, quantization levels, consistency loss weights, step sizes, and, in generalized frameworks, parameters for ODE solvers and re-noising schedules. While parameter sensitivity is moderate, optimal performance requires validation set tuning.

When operating under high-noise (early) timesteps, information content is minimal, and bit allocation may be wasted unless adaptive schedules are deployed. Potential future advancements include adaptive time-step skipping, learned codebooks per step, and extension to conditional, text-guided, or non-image modalities (audio, video).

Experimental results across image compression, restoration, multi-exposure fusion, and editing consistently show that DDCMs—and in particular, variants enforcing or leveraging cross-step consistency—yield improved trade-offs in perceptual quality, distortion, and rate, surpassing non-consistent or baseline diffusion approaches (Kong, 17 Nov 2025, Vaisman et al., 9 Nov 2025, Cheng et al., 2024, Fabian et al., 2023, Bemana et al., 2024, Xu et al., 2023).

---

References:

• "Generalized Denoising Diffusion Codebook Models (gDDCM): Tokenizing images using a pre-trained diffusion model" (Kong, 17 Nov 2025)
• "Consistent Diffusion: Denoising Diffusion Model with Data-Consistent Training for Image Restoration" (Cheng et al., 2024)
• "Turbo-DDCM: Fast and Flexible Zero-Shot Diffusion-Based Image Compression" (Vaisman et al., 9 Nov 2025)
• "DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency" (Fabian et al., 2023)
• "Bracket Diffusion: HDR Image Generation by Consistent LDR Denoising" (Bemana et al., 2024)
• "Inversion-Free Image Editing with Natural Language" (Xu et al., 2023)
• "Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent" (Daras et al., 2023)