PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression

Abstract: We present PolarQuant, a post-training weight quantization method for LLMs that exploits the distributional structure of neural network weights to achieve near-lossless compression. PolarQuant operates in three stages: (1) block-wise normalization to the unit hypersphere, (2) Walsh-Hadamard rotation to transform coordinates into approximately Gaussian random variables, and (3) quantization with centroids matched to the Gaussian distribution. Our ablation reveals that Hadamard rotation alone accounts for 98% of the quality improvement, reducing Qwen3.5-9B perplexity from 6.90 (absmax Q5) to 6.40 (Delta = +0.03 from FP16), making it practically lossless without any calibration data. Furthermore, PolarQuant functions as an effective preprocessing step for downstream INT4 quantizers: PolarQuant Q5 dequantized and re-quantized by torchao INT4 achieves perplexity 6.56 versus 6.68 for direct absmax INT4, while maintaining 43.1 tok/s throughput at 6.5 GB VRAM. Code and models are publicly available.

• The paper presents a novel quantization pipeline that uses Hadamard rotation to transform LLM weight distributions to nearly Gaussian, enabling MSE-optimal Lloyd-Max quantization.
• It achieves significant perplexity improvements by reducing Qwen3.5-9B perplexity from 6.90 to 6.39 in Q5 format while operating with negligible computational overhead.
• The method is calibration-free, computationally efficient, and composable with subsequent quantizers, making it suitable for deployment on constrained hardware.

PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression

Introduction and Motivation

PolarQuant presents a post-training quantization scheme targeting the efficient deployment of LLMs on constrained hardware by aggressively compressing weight tensors. The method exploits the empirical observation that, after block-wise normalization and appropriate orthogonal transformation, the distribution of LLM weights closely approximates a standard normal. Classic absmax quantization, though widely adopted, is mismatched to these empirical weight distributions, as it wastes codebook levels on infrequent outlier magnitudes and concentrates quantization error in dense central regions. While alternative quantizers (GPTQ, AWQ, NF4, QuIP#) attempt distributional adaptation or use second-order statistics, PolarQuant leverages a theoretically principled pipeline designed to maximize MSE-optimality for Gaussian-distributed weights, while remaining computationally efficient and calibration-free.

Methodology

PolarQuant operates in four key stages: (1) block-wise $\ell_2$ normalization to the unit sphere, (2) intra-block normalized Walsh-Hadamard rotation that homogenizes the block distribution, (3) scaling to unit variance, and (4) quantization via Lloyd-Max centroids optimal for $\mathcal{N}(0,1)$. The process is invertible, with negligible storage overhead (only per-block norms and a centroid table). The Hadamard matrix, due to orthogonality and symmetry, acts as its own inverse and demands minimal memory.

The critical insight is that Hadamard rotation spreads any outlier structure uniformly across coordinates, invoking the central limit theorem: this ensures marginal elements post-rotation are near-Gaussian even for moderate block sizes ($d=128$). Consequently, the per-block quantization problem is reduced to classical scalar quantization for a standard normal, solvable with closed-form Lloyd-Max procedures. Notably, the symmetry of the standard normal reduces centroid storage cost. The method accommodates composability, serving both as a direct quantizer and as distributional preprocessing for second-stage quantizers such as INT4.

Empirical Results

Evaluation on the Qwen3.5-9B model yields a notable perplexity improvement using PolarQuant Q5: perplexity drops from 6.90 (absmax Q5) to 6.39, incurring a negligible delta ($+0.02$) from FP16. Ablation studies indicate that Hadamard rotation is overwhelmingly responsible for this gain, accounting for 98% of loss reduction at Q5; Lloyd-Max centroids contribute marginally at standard bitwidths. This implies that the dominating failure mode of classic quantization is the distribution mismatch, not suboptimal centroid allocation for Gaussian sources—an important insight for the design of future quantization strategies.

Further, PolarQuant excels as a preprocessing module: when the Q5 dequantized weights are subsequently quantized by torchao INT4, perplexity stands at 6.56 versus 6.68 for native absmax INT4. This result holds at equivalent throughput and memory consumption, with cross-platform confirmation on Apple Silicon (Mac mini M4, 19.7 tok/s at 4.8 GB). PolarQuant combined with AWQ reduces perplexity further (6.43), but uniform Q5 is found superior to AWQ with mixed-bit allocation, rendering the pipeline simple and robust.

A comprehensive analysis of variants clarifies that cascading quantization works optimally when the initial quantizer preserves sufficient signal: overly aggressive (e.g., Q3) intermediate quantization irretrievably degrades quality, resulting in inferior downstream INT4 results. The algorithm introduces only a minimal one-time dequantization overhead (about 8 seconds at load time) with zero runtime penalty.

Theoretical Considerations and Algorithmic Insights

The use of the Hadamard transform is theoretically motivated by its deterministic orthogonal structure, maximally mixing block energies and ensuring marginal Gaussianity in projected space. This is tighter than random rotation (as in QuIP#), is trivial to implement with $O(d \log d)$ complexity, and introduces no additional parameters. The Lloyd-Max solution for the Gaussian source is well-analyzed, with closed-form centroid and boundary updates that converge rapidly.

By demonstrating that nearly all quantization error reduction comes from the Hadamard-induced distributional homogenization, the work highlights that the error landscape for LLM quantization is governed more by alignment to the correct statistical model than by centroid quantizer optimality per se.

Implications and Future Directions

Practically, PolarQuant unlocks the ability to run larger LLMs on consumer hardware (NVIDIA RTX, Apple Silicon) with negligible loss in perplexity, transparent integration into existing pipelines (torchao, GGUF, MLX), and no need for calibration data. The method’s minimal per-block overhead is amortized by the reduction in quantization error and enables storage of high-fidelity models useful both for inference and as intermediate representation before more aggressive quantization.

Theoretically, the results advocate for a shift in quantizer design: aligning the quantizer to the latent (potentially transformed) distribution of weights can dominate the error budget. It remains open to investigate scenarios in which the original weight distribution diverges from near-normal—e.g., alternative architectures, highly structured models, or atypical training regimes. Extending this framework to activation quantization, mixed precision schemes, or enabling joint optimization of block rotation and centroid spacing under task-specific objectives could yield further gains.

Conclusion

PolarQuant introduces an optimal, theoretically justified quantization pipeline for LLM compression, melding efficient Hadamard-based distribution alignment with classical scalar quantization. The empirical results validate its efficacy, with near-lossless compression and simple, portable implementation. The work establishes that orthogonal rotation, not merely centroid optimization, underpins effective quantization for modern LLMs, inviting further study into optimal representation alignment for DNN parameter spaces.