Walsh-Hadamard Transform Overview

Updated 28 January 2026

Walsh-Hadamard Transform is a real, orthogonal spectral transform that uses ±1 piecewise-constant basis functions to effectively represent abrupt, binary-valued signals.
It employs fast, in-place algorithms based solely on additions and subtractions, achieving O(n log n) complexity for efficient computation in various fields.
The transform is pivotal in applications such as neural operators, quantum simulation, and compressed linear algebra, offering robust handling of discontinuities without Gibbs phenomena.

The Walsh-Hadamard Transform (WHT) is a real, orthogonal, non-sinusoidal spectral transform whose basis functions—the Walsh functions—are piecewise-constant rectangular waves taking only ±1 values. The WHT provides a computationally efficient alternative to the Fourier transform for many settings, particularly where representation of abrupt, piecewise-constant, or binary-valued structures is essential. It admits fast, in-place algorithms exclusively using additions and subtractions, admits substantial theoretical generalization, and finds application across signal processing, neural computation, scientific computing, compressed linear algebra, and quantum simulation.

1. Mathematical Definition and Fundamental Properties

Let $n = 2^k$ for integer $k \geq 0$ . The (unnormalized) Hadamard matrix $H_n \in \{+1, -1\}^{n \times n}$ is defined recursively: $H_1 = [1], \quad H_{2n} = \begin{bmatrix} H_n & H_n \ H_n & -H_n \end{bmatrix}$ The normalized Hadamard matrix $\hat{H}_n = H_n / \sqrt{n}$ is orthonormal ( $\hat{H}_n \hat{H}_n^T = I_n$ ). For a real vector $x \in \mathbb{R}^n$ , its Walsh-Hadamard transform is $\hat{y} = \hat{H}_n x$ , and the inverse is identical. The 2D WHT for a field $U \in \mathbb{R}^{h \times w}$ , with $h$ and $w$ both powers of two, is given by $\hat{U} = \hat{H}_h U \hat{H}_w^T$ . The Walsh basis functions are orthonormal piecewise-constant (rectangular) waveforms with a specific number of zero crossings (“sequency”), providing a natural analog to frequency in the sinusoidal Fourier basis. For discrete indices, $[H_n]_{i,j} = (-1)^{\langle i, j \rangle}$ where $\langle i, j \rangle$ indicates the bitwise dot-product modulo 2 (Cavallazzi et al., 10 Nov 2025).

The WHT is self-inverse (up to normalization), and its basis is mutually orthogonal: $\int_0^1 w_i(t) w_j(t) \, dt = 0, \quad \text{for} \ i \ne j$ This structure enables succinct, non-oscillatory representation of discontinuities.

2. Fast Walsh-Hadamard Transform Algorithms

The standard Fast Walsh-Hadamard Transform (FWHT) is a divide-and-conquer butterfly algorithm structurally analogous to the Cooley–Tukey FFT, but requiring only additions and subtractions:

for half_length = 1 to n/2 by doubling:
    for start = 0 to n-1 by 2*half_length:
        for i = 0 to half_length-1:
            u = x[start + i]
            v = x[start + i + half_length]
            x[start + i] = u + v
            x[start + i + half_length] = u - v

This routine applies recursively to vectors of length

n = 2^k

with total arithmetic complexity

O(n \log n)

(Cavallazzi et al., 10 Nov 2025, Jeong et al., 2019). The 2D transform applies the FWHT to rows and then columns, yielding

O(hw \log(hw))

for matrices.

Variants extend the basic algorithm. The lookup-table–augmented WHT enables $O(N \log N / \log\log N)$ bit complexity by “skipping” several recursion levels via precomputed short-length transforms, a significant improvement for field operations over constantsized finite fields (Alman, 2022). The Cascading Haar Wavelet algorithm re-expresses the WHT as a recursive cascade of Haar transforms, offering a different route to $O(n \log n)$ complexity and regular parallelization (Thompson, 2016). For ultra-large $N$ , external-memory and distributed implementations scale the FWHT to tera-scale data (Lu, 2016).

3. Walsh Functions: Relation to Fourier Bases and Discontinuity Representation

Walsh functions form a complete, piecewise-constant orthonormal basis on $[0,1]$ , indexed by sequency (number of sign-changes per interval), providing a sharp contrast to the oscillatory nature of Fourier modes (Cavallazzi et al., 10 Nov 2025). Their abrupt structure means step or piecewise constant functions can be represented exactly by a small subset of these basis functions, with no Gibbs phenomenon. For example, a step signal is represented by a finite sum of Walsh functions whose sequency indexes align with discontinuity positions, producing rapid spectral decay and allowing severe coefficient truncation without loss of fidelity.

The absence of localized frequency yields highly localized features, making these bases extremely suitable for representing fields with sharp spatial transitions (e.g., binary, blocky, or piecewise-constant signals) where Fourier techniques underperform due to Gibbs artifacts (Cavallazzi et al., 10 Nov 2025). This property has led to the introduction of Walsh-Hadamard Neural Operators (WHNO), which outperform classic Fourier-based models on PDEs with discontinuous coefficients.

4. Algorithmic Generalizations, Efficient Implementations, and Matrix Structure

Several generalizations and optimizations exist for the WHT:

Sparse and noisy-sparse transforms: When only a sublinear ( $K = O(N^\alpha), \alpha < 1$ ) fraction of the spectral coefficients are nonzero, belief-propagation/peeling-style algorithms using hashing, subsampling, and aliasing enable reconstruction of sparse WHTs in $O(K \log(N/K)\log K)$ time (Scheibler et al., 2013).
Hardware acceleration: HadaCore demonstrates a mapping of the FWHT onto NVIDIA GPU Tensor Cores, refactoring the butterfly recursion to maximize matrix–matrix multiply–accumulate (MMA) throughput. This typically delivers empirical speedups of $1.1\times$ to $3.6\times$ over CUDA-core implementations with no precision loss, even with quantized arithmetic (Agarwal et al., 2024).
Further speedups via matrix non-rigidity: Refined algorithms reduce the leading constant to $23/24\,N \log N + O(N)$ for the total arithmetic operation count by decomposing the Hadamard matrix into a low-rank plus sparse sum and recursing on blocks of size 8 (Alman et al., 2022).
Block and multiallelic generalizations: The WHT extends beyond binary; the multiallelic WHT provides transforms for arbitrary numbers of alleles per site, yielding dual, degree-block-diagonal transforms that recover the classical binary case for $k_i=2$ (Greene, 2023).
Algorithmic structure space: The set of all FWHT dataflows is characterized as sequences of “butterfly” arrays interleaved with linear (bit-matrix) permutations, opening a vast design space for hardware or locality optimization (Serre et al., 2017).

5. Applications Across Scientific and Computational Domains

Neural operators and PDEs: WHT-based neural operators (WHNOs) have demonstrated better preservation of sharp solution features and lower error exhibiting 24–38% improvements for piecewise-constant PDE coefficients versus Fourier-based neural operators, with up to 40% error reduction when ensembled with FNOs (Cavallazzi et al., 10 Nov 2025).
Neural network inference: Replacing parametric $1\times1$ convolutions in CNNs by fixed WHT transforms yields substantial FLOP and parameter reductions with equal or improved accuracy. On MobileNet-V1, using DWHT-based pointwise layers resulted in a 79.1% reduction in parameters, a 48.4% FLOP reduction, and a modest (+1.49%) accuracy increase (Jeong et al., 2019). Smooth-thresholding WHT domains have reduced parameter counts in MobileNet-V2 by over 50% with only small accuracy drop, and have exhibited 2x runtime speedups vs learned $1\times1$ layers (Pan et al., 2021). DWHT-based 2D layers can replace $3\times3$ convolutions with substantial runtime (24x), parameter, and memory savings (Pan et al., 2022).
Quantum simulation and Pauli operator decomposition: The Pauli decomposition of $N\times N$ Hermitian matrices can be performed in exact $O(N^2 \log N)$ time and constant extra space by permuting rows (“XOR step”), applying FWHT to each row, and a diagonal phase correction, enabling efficient generation of Pauli-sum representations for quantum algorithms and outperforming prior C++ and Qiskit implementations (Georges et al., 2024).
Hybrid quantum-classical algorithms: Quantum circuits composed of $\log_2 N$ Hadamard gates enable $O(N)$ -complexity 1D or 2D WHTs for image processing; this approach can outperform $O(N \log N)$ classical FWHT in scenarios with efficient quantum state preparation (Rohida et al., 2024, Shukla et al., 2022).
Compressed and randomized linear algebra: In compressed matrix multiplication, substituting the FFT with the FWHT in sketching (XOR-convolution) preserves unbiasedness and variance but yields practical 3-4x speedups in favorable sparsity regimes (Andersson et al., 14 Jan 2026).
Genetic and evolutionary computation: The multiallelic WHT enables direct, block-diagonal representation of higher-order interactions in arbitrary allele-count landscapes, with applications in genetic epistasis analysis (Greene, 2023).

6. Practical and Theoretical Considerations

The appeal of the WHT arises from several factors:

Multiplication-free arithmetic: Its kernel consists entirely of ±1, allowing algorithms to be implemented with only addition and subtraction, beneficial for hardware efficiency and low-power or quantized arithmetic (Jeong et al., 2019, Agarwal et al., 2024).
Memory and parallelization: The FWHT is an in-place algorithm with predictable streaming memory access, suitable for both multicore and architecture-unaware parallelization. Distributed large-scale implementations enable tera-scale transforms for data-intensive science (Lu, 2016).
Spectral compression and noise robustness: In tasks dominated by piecewise-constant features, aggressively truncating the spectral expansion yields near-lossless approximation, with most energy in low-sequency modes and exact step representation (Cavallazzi et al., 10 Nov 2025).
Hybrid domain application: Ensembles of WHT and Fourier bases are effective in mixed smooth–discontinuous problems, leveraging the complementarity of rectangular Walsh and sinusoidal Fourier modes. For quantization-aware adaptation in LLMs, the WHT’s piecewise basis achieves better quantization error reduction than DCT/DHT or low-rank SVD, with the fewest trainable parameters, and simplified adapter initialization (Jeon et al., 22 Sep 2025).

7. Limitations, Extensions, and Open Directions

Input size constraints: Traditional algorithms require $n$ to be a power of two; block and multiallelic formulations mitigate this but may require padding or per-block operations.
Hardware-specific optimization: For ultimate performance on GPUs or FPGAs, low-level tuning is required (e.g., HadaCore), and the comparative benefit versus heavily optimized FFT decreases with input smoothness or lack of discontinuities (Agarwal et al., 2024).
Sparse transform recovery: While exact sparse WHTs can be obtained efficiently under random (or generic) support, robust handling of heavily noisy or “nuisance” coefficients at massive scale remains challenging (Scheibler et al., 2013, Lu, 2016).
Non-binary generalization: For multiallelic settings, dual transforms are required, each block-diagonal by interaction degree, and the conventional ±1 orthogonality becomes more intricate due to the broadened allele support (Greene, 2023).
Enumeration and structural diversity: The space of valid FWHT algorithms is large, admitting extensive hardware- and application-specific customization in dataflow schemes (Serre et al., 2017).

Major references:

"Walsh-Hadamard Neural Operators for Solving PDEs with Discontinuous Coefficients" (Cavallazzi et al., 10 Nov 2025)
"New pointwise convolution in Deep Neural Networks through Extremely Fast and Non Parametric Transforms" (Jeong et al., 2019)
"A Fast Hadamard Transform for Signals with Sub-linear Sparsity in the Transform Domain" (Scheibler et al., 2013)
"Pauli Decomposition via the Fast Walsh-Hadamard Transform" (Georges et al., 2024)
"The Cascading Haar Wavelet algorithm for computing the Walsh-Hadamard Transform" (Thompson, 2016)
"Faster Walsh-Hadamard Transform and Matrix Multiplication over Finite Fields using Lookup Tables" (Alman, 2022)
"HadaCore: Tensor Core Accelerated Hadamard Transform Kernel" (Agarwal et al., 2024)
"Hybrid classical-quantum image processing via polar Walsh basis functions" (Rohida et al., 2024)
"Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks" (Pan et al., 2021)
"Practical Tera-scale Walsh-Hadamard Transform" (Lu, 2016)
"Faster Walsh-Hadamard and Discrete Fourier Transforms From Matrix Non-Rigidity" (Alman et al., 2022)
"Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networks" (Pan et al., 2022)
"A hybrid classical-quantum algorithm for digital image processing" (Shukla et al., 2022)
"Engineering Compressed Matrix Multiplication with the Fast Walsh-Hadamard Transform" (Andersson et al., 14 Jan 2026)
"Characterizing and Enumerating Walsh-Hadamard Transform Algorithms" (Serre et al., 2017)
"QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on LLMs" (Jeon et al., 22 Sep 2025)
"Multiallelic Walsh transforms" (Greene, 2023)