Activation-Quantization-Aware Scaling (AQAS)

• AQAS is a suite of joint quantization techniques that calibrate and balance neural network weights and activations to minimize mean-squared error.
• It employs calibration, analytic scaling, and per-channel/per-bit strategies to harmonize dynamic ranges across modern deep learning architectures.
• Empirical results demonstrate that AQAS improves model fidelity with minimal overhead, achieving near full-precision performance in language and vision tasks.

Activation-Quantization-Aware Scaling (AQAS) is a suite of methodologies developed to optimize joint quantization of neural network weights and activations, minimizing distortion in highly compressed deep learning models and LLMs. AQAS encompasses calibration, training, and analytic scaling strategies that explicitly balance weight and activation dynamic ranges within quantization pipelines, yielding improved model fidelity and hardware efficiency across diverse architectures and quantization regimes.

1. Fundamental Principles and Problem Setting

High-performance neural networks increasingly employ quantization to achieve drastic reductions in memory and compute costs. Conventional post-training quantization (PTQ) approaches typically address weight and activation scaling in isolation, via either weight-only quantization (e.g., OPTQ, AWQ) or activation scaling techniques (e.g., SmoothQuant). However, modern architectures such as OPT and LLaMA exhibit sharply divergent dynamic-range characteristics in their weights and activations, leading to conflicting constraints under asymmetric quantization (e.g., W4A8). As a result, isolated scaling produces severe error in one operand when tuned for the other.

AQAS resolves this paradigm conflict by introducing joint scaling across channels, designed to minimize the mean-squared output distortion of quantized matrix multiplication. Formally, for $Y = WX$ (with $W \in \mathbb{R}^{M \times C}$, $X \in \mathbb{R}^{C \times T}$), AQAS seeks channel-wise scales $\mathbf{s} \in \mathbb{R}^C$ that minimize: $$s^* = \operatorname*{argmin}_{s} \| Q_W(W \mathrm{diag}(s))\, Q_X(\mathrm{diag}(s)^{-1} X) - WX \|_2^2$$ where $Q_W(\cdot)$ and $Q_X(\cdot)$ denote uniform (typically asymmetric) quantizers at the target bit-widths (Lee et al., 2023).

2. AQAS Calibration, Optimization, and Implementation

The AQAS calibration workflow operates at the linear layer or fused block level (e.g., fully connected + layer normalization). The procedure comprises:

• Sample Collection: Gathering activation samples and weights via forwarding calibration sequences through the unquantized model.
• Statistic Calculation: Channel-wise determination of maxima (max-based statistics yield robust quantization, as demonstrated in Table A.7 of (Lee et al., 2023)).
• Search Grid Specification: Defining candidate scaling factors $s_c$ (commonly powers of two or fine grids near unity).
• Error Accumulation: For each candidate scaling vector, quantizing and evaluating the batch output error.
• Scale Selection and Freezing: Choosing $s$ minimizing accumulated mean-squared error (MSE), imposing bounds for stability, and propagating these static scales into inference kernels.

This layer-wise strategy maintains negligible runtime overhead, given the scale adjustments are fused into existing scale-multiply operations, incurring no additional integer multiplications. No datapath changes are necessary for INT4–INT8 or related MAC units; calibration completes in seconds per layer.

3. Mathematical Formulation—Analytic Scaling and Harmonic Minimization

Several variants of AQAS propose analytic solutions for harmonizing quantization difficulty between weights and activations. In HarmoQ, per-layer scaling $s^*$ is derived to equilibrate activation and weight MSE:

$$\begin{align*} \mathrm{MSE}_x(s) &= \frac{(\beta_x - \alpha_x)^2}{12\,s^2\,(2^{b_x}-1)^2}, \ \mathrm{MSE}_w(s) &= \frac{(\beta_w - \alpha_w)^2\,s^2}{12\,(2^{b_w}-1)^2}, \ s^* &= \sqrt{\frac{(\beta_x - \alpha_x)\,(2^{b_w}-1)}{(\beta_w - \alpha_w)\,(2^{b_x}-1)}} \end{align*}$$

where $(\alpha_x, \beta_x), (\alpha_w, \beta_w)$ are clipping bounds for activations and weights, $(b_x, b_w)$ bit-widths, and $s^*$ is computed per layer or channel (Wang et al., 8 Nov 2025). Iterative refinement is achieved by embedding this closed-form scaling within a loop of structural residual calibration and adaptive boundary refinement, maintaining control over activation-induced detail loss and weight-induced texture artifacts.

4. Per-Channel and Per-Bit Scaling Schemes

AQAS admits both per-channel and per-bit implementations, depending on the regime:

• Per-channel (as in GranQ (Hong et al., 24 Mar 2025)): Scale and zero-point vectors $\vec{s}, \vec{z}$ computed from per-channel statistics; quantization is fully vectorized and broadcast via modern tensor frameworks, supporting scalable zero-shot quantization and efficient QAT.
• Per-bit scaling (see (Song et al., 7 Apr 2025)): High-resolution quantization (e.g., INT4 activation as $4 \times \mathrm{INT1}$ planes) is decomposed into multiple Boolean planes, each with its own scale. Smoothing and tuning of these scales via regularized least-squares (coordinate descent) minimizes quantization error while controlling for overfitting.

Across such implementations, AQAS adapts the quantization process to minimize both clipping and rounding errors while maximally utilizing available quantization bins (cf. ASQ (Zhou et al., 24 Apr 2025), per-adapter step size learning).

5. Empirical Results and Scaling Law Implications

Comprehensive experiments validate AQAS efficacy:

• Language Modeling Tasks: PTQ with AQAS recovers near-FP16 perplexities for large LLMs, e.g., OPT-6.7B: AQAS+OPTQ at 12.97 vs. FP16 12.29; LLaMA-7B: 5.71 vs. FP16 5.68 (Lee et al., 2023).
• Zero-Shot and In-Context Learning: AQAS consistently yields $<1\%$ accuracy loss versus full-precision, outperforming weight-only and activation-only scaling by up to 15% (Lee et al., 2023).
• Vision Models: GranQ achieves up to $+5.45\%$ accuracy gain in 3-bit settings on CIFAR-100, with latency overhead $\lesssim 10\%$ (Hong et al., 24 Mar 2025). HarmoQ offers $+0.46$ dB to $+0.66$ dB PSNR advantage over state-of-the-art on 2–3 bit super-resolution (Wang et al., 8 Nov 2025).

Table: Effective Parameter Multipliers (EPM) for Full Quantization (Frantar et al., 23 Feb 2025) | Bit-width | EPM | Size Gain | |----------:|:-----:|:----------:| | 8-bit | 0.857 | 1.17× | | 4-bit | 0.747 | 1.34× | | 2-bit | 0.289 | 3.46× | | 1-bit | 0.067 | 14.93× |

In scaling-law analyses, joint AQAS quantization modifies the "effective" parameter budget multiplicatively in relation to sparsity, guiding model and hardware design. For example, 8w8a quantization achieves parameter efficiency comparable to 50% sparsity; 4w4a outperforms 75% sparsity (Frantar et al., 23 Feb 2025).

6. Extensions, Limitations, and Best Practices

AQAS is robust across bit-widths, quantization configurations, and neural architectures, but practical limitations include:

• Dependence on Data Statistics: Calibration statistics from synthetic or real data may dictate optimal scaling, with dynamic or learnable scaling variants addressing distributional shifts (Hong et al., 24 Mar 2025, Zhou et al., 24 Apr 2025).
• Low-Bit Regimes: Model quality degrades sharply below 4 bits for activations, with empirical EPM curves showing diminishing returns (Frantar et al., 23 Feb 2025).
• Phased and Static Scaling: SASQ demonstrates that optimizing only static activation scales (clamp-centric QAT) can yield accuracy above FP16 baselines without retraining weights (Mao et al., 16 Dec 2025).
• Mixed Precision, Adaptive Schemes: Decomposition of activations into bit-planes and use of regularized scale smoothing enable robust binarization (Song et al., 7 Apr 2025). Mixed-precision clustering and dynamic calibration represent open research avenues.

7. Integration in Quantization-Aware Training and Hardware Pipelines

AQAS strategies integrate seamlessly into PTQ and QAT pipelines. They are implemented with negligible compute overhead—vectorized scaling and quantization operations leverage GPU primitives for optimal performance. In hardware, AQAS enables efficient integer MAC utilization, and in some cases introduces hybrid data formats (e.g., dINT) for underflow mitigation (Lee et al., 2023).

The unification of sparsity and quantization scaling laws further provides a principled framework for selecting compression configurations that balance compute budget, model quality, and deployment efficiency (Frantar et al., 23 Feb 2025). AQAS thus underpins state-of-the-art quantization solutions in modern deep learning, maximizing accuracy retention and resource savings under aggressive compression.