Q-DiT4SR: Efficient PTQ for DiT Super-Resolution

• Q-DiT4SR is a post-training quantization framework that uses hierarchical SVD and variance-aware mixed precision to preserve high-frequency details in DiT-based super-resolution.
• It decomposes weights into a global low-rank branch and a local block-wise rank-1 branch, effectively balancing broad structural integrity with fine granular features.
• By dynamically allocating bit-widths for both weights and activations, Q-DiT4SR reduces computational cost (over 60× FLOPs reduction) while maintaining near full-precision perceptual quality.

Q-DiT4SR is a post-training quantization (PTQ) framework designed to enable efficient, detail-preserving deployment of DiT4SR—an advanced Diffusion Transformer architecture for real-world image super-resolution (Real-ISR). Q-DiT4SR introduces a hierarchical low-rank decomposition and a variance-aware, spatio-temporal mixed precision quantization scheme tailored to the high sensitivity of DiT-based super-resolution to quantization artifacts. It achieves near full-precision perceptual quality at substantially reduced model size and computational cost, representing the first PTQ solution specifically engineered for Diffusion Transformers in Real-ISR (Zhang et al., 1 Feb 2026).

1. Quantization Challenges in DiT-based Real-World Image Super-Resolution

Diffusion Transformers (DiTs), such as DiT4SR, offer state-of-the-art performance in perceptual fidelity for Real-ISR tasks, but are characterized by large parameter counts (up to billions), high FLOPs, and a dependence on fine high-frequency features that are highly susceptible to quantization-induced distortion. Existing PTQ methods are predominantly U-Net–centric or optimized for text-to-image generation; direct application of these to DiT4SR leads to severe local texture degradation (Zhang et al., 1 Feb 2026).

Three quantization obstacles are central:

• Quantization error can accumulate and compound over the hundreds of iterative denoising steps characteristic of diffusion inference, producing amplified fidelity loss.
• Activation statistics shift dramatically across both layers and diffusion timesteps, undermining uniform or static bit-width assignments.
• Global low-rank matrix approximations (e.g., single-branch SVD) are insufficient for capturing the fine-grained residual structure vital for photorealism in Real-ISR, especially in DiT-based architectures (Zhang et al., 1 Feb 2026).

2. Hierarchical SVD Decomposition for Weight Quantization

Q-DiT4SR addresses the inadequacy of global-only low-rank methods through the introduction of Hierarchical SVD (H-SVD), which combines:

1. Global Low-Rank Branch (SVD-G): For a weight matrix $W\in\mathbb R^{\mathrm{out}\times \mathrm{in}}$, a rank-$r$ SVD after Hadamard normalization yields $W_{H} \approx W_{\mathrm{SVD-G}} = AB$, $A\in\mathbb{R}^{\mathrm{out} \times r}$, $B\in\mathbb{R}^{r \times \mathrm{in}}$.
2. Local Block-Wise Rank-1 Branch (SVD-L): The residual $W_{\mathrm{res}}=W_H-W_{\mathrm{SVD-G}}$ is partitioned into non-overlapping blocks, each of which is approximated by its principal singular component: $\hat W^{(p,q)} = \sigma_{p,q}u_{p,q}v_{p,q}^\top$. All blocks are reassembled to form $W_{\mathrm{SVD-L}}$.

By tuning the block size and global rank so that the total parameter budget equals that of a pure rank-$r$ SVD, H-SVD preserves both broad and fine-grained structure (Zhang et al., 1 Feb 2026). The quantized forward pass reconstructs activations as:

$$\hat{W} = (W_{\mathrm{SVD-G}} + W_{\mathrm{SVD-L}}) H_n^\top + Q_w(W_{\mathrm{res}} - W_{\mathrm{SVD-L}}) H_n^\top$$

where $Q_w(\cdot)$ denotes per-channel symmetric quantization. This joint scheme aligns closely to the original spectral energy and empirically preserves high-frequency detail critical for Real-ISR (Zhang et al., 1 Feb 2026).

3. Variance-Aware Mixed Precision Bit-Width Allocation

Statistical analysis reveals that quantization distortion for a Gaussian source scales with the product of its variance and the exponential of the bit-width. Both weights and activations (after Hadamard transform) are approximated as Gaussian, motivating variance as the bit allocation criterion (Zhang et al., 1 Feb 2026).

3.1. Variance-Aware Spatio Mixed Precision (VaSMP) – Weights

For each layer $\ell$ with $N_\ell$ weights and average output-channel variance $\bar\sigma^2_\ell$, VaSMP solves:

$$\min_{\{b_\ell\}} \sum_\ell N_\ell \bar\sigma^2_\ell 2^{-2b_\ell} \quad \text{s.t.} \quad \sum_\ell N_\ell b_\ell = B_\mathrm{target}$$

The unconstrained relaxed solution is:

$$b^*_\ell = B_\mathrm{target} + \frac{1}{2}\left[ \log_2 \bar\sigma^2_\ell - \overline{\log_2\bar\sigma^2} \right]$$

where the overline denotes a parameter-weighted average across all layers. Bit-widths are rounded and greedily assigned to maximize variance reduction per bit, and no weight calibration data are needed (Zhang et al., 1 Feb 2026).

3.2. Variance-Aware Temporal Mixed Precision (VaTMP) – Activations

For activations at layer $\ell$ and diffusion timestep $t$ with mean variance $v_{\ell,t}$, and for candidate bit-width $b$ with normalized distortion $\kappa(b)$, VaTMP minimizes:

$$\min_{\{b_{\ell,t}\}} \sum_{t=1}^{T_\ell} v_{\ell,t} \kappa(b_{\ell,t}) \quad \text{s.t.} \quad \sum_{t=1}^{T_\ell} b_{\ell,t} \leq B_\ell$$

A dynamic programming algorithm segments the timestep sequence into piecewise-constant intervals, assigning higher bits to high-variance timesteps and reducing precision where activations are more stable. Only minimal calibration data (e.g., 32 low-resolution crops) are required for variance estimation (Zhang et al., 1 Feb 2026).

4. Numerical Benchmarks and Perceptual Metrics

Q-DiT4SR is evaluated on DiT4SR backbones with $4\times$ upsampling across multiple real-world datasets: DrealSR, RealSR, RealLR200, and RealLQ250. Only perceptual and no-reference metrics are reported: LPIPS (↓), MUSIQ (↑), MANIQA (↑), CLIPIQA (↑), and LIQE (↑).

Configuration	Parameter Compression	FLOPs Reduction	Visual/Perceptual Quality (vs. Full Precision)

W4A6 (weights in {4,6,8} bits, activations 6 bits) | - | - | Matches or exceeds full-precision (LPIPS 0.383 vs. 0.3897) W4A4 (weights in {4,6,8} bits, activations 4 bits) | 5.8× | >60× | Best LPIPS/LIQE, MUSIQ/MANIQA +1–3 points over alternatives

Under both W4A6 and W4A4, Q-DiT4SR substantially outperforms prior PTQ techniques (SVDQuant, Q-DiT, PTQ4DiT), particularly in the low bit-width setting where competitors exhibit significant degradations in perceptual and textural quality (Zhang et al., 1 Feb 2026). Qualitative results indicate sharper edges and finer textures (e.g., in foliage and bricks) compared to uniform and single-branch decompositions.

5. Analysis of Component Contributions

Ablation experiments elucidate the impact of major Q-DiT4SR components:

• H-SVD Local Branch Size: Increasing local budget up to 8 blocks improves MUSIQ (from 66.71 to 67.72) with minimal extra FLOPs, but offers diminishing or negative returns beyond that.
• VaSMP vs. Uniform/MSE-Based Bit Allocation: H-SVD+VaSMP achieves best MUSIQ under W4A6, outperforming flat 4-bit and naive mixed-precision policies.
• VaTMP Temporal Scheduling: Addition of VaTMP yields further MUSIQ gains (+0.53 under W4A4), as finer-grained bit allocation across timesteps sharpens local image details (e.g., hair, textures).

The joint application of H-SVD, VaSMP, and VaTMP is necessary for the preservation of high-frequency structure under aggressive quantization constraints; omitting any component induces a measurable loss in fidelity or an increase in perceptual artifacts (Zhang et al., 1 Feb 2026).

6. Significance, Limitations, and Context

Q-DiT4SR establishes the first PTQ protocol specifically targeting the deployment constraints and error-sensitivity of DiT-based super-resolution models, achieving compression and acceleration factors previously unattainable without severe quality loss. Requiring zero calibration data for weights and only minimal samples for temporal activation scheduling, it enables resource-constrained inference while maintaining state-of-the-art perceptual realism (Zhang et al., 1 Feb 2026).

A plausible implication is that further research into structured decompositions and data-free quantization could generalize these methodologies to other iterative generative architectures with variable activation statistics. However, Q-DiT4SR presupposes access to the architectural details of DiT4SR and may be limited in applicability to highly different architectures or tasks with fundamentally different input domains.

References:

Q-DiT4SR: "Q-DiT4SR: Exploration of Detail-Preserving Diffusion Transformer Quantization for Real-World Image Super-Resolution" (Zhang et al., 1 Feb 2026) DiT4SR: "DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution" (Duan et al., 30 Mar 2025)