Constructive Approximation of Multiplication

• The paper introduces a deterministic quadratic-time algorithm for approximate matrix multiplication with a guaranteed Frobenius-norm error bound, utilizing iterative solvers and low-rank projections.
• Neural network constructions approximate multiplication with precise Sobolev error control using GELU activation and indicator-neuron architectures, enabling scalable and modular designs.
• Bilinear, FFT-based, and operator-projection techniques balance arithmetic overhead and error propagation to achieve efficient, large-scale multiplication approximations.

Constructive approximation of multiplication refers to explicit, algorithmically realizable methods for approximating the product of two quantities—scalars, vectors, matrices, or operators—to controlled error, frequently in settings where exact multiplication is computationally expensive or analytically unwieldy. This topic spans deterministic and randomized algorithms for matrix multiplication, neural network multiplication blocks with quantifiable Sobolev error, operator-theoretic finite-dimensional projections, and specialized fast polynomial or bilinear schemes for integer and matrix multiplication. The field addresses both the algorithmic construction (how the approximation is produced) and precise error bounds in suitable operator, normed, or derivative spaces.

1. Deterministic Quadratic-Time Matrix Multiplication via Linear Systems

Manne and Pal (Manne et al., 2014) introduce a constructive quadratic-time deterministic algorithm for approximate multiplication of two $n\times n$ matrices $A, B$, returning $C'$ such that $\|C' - AB\|_F \leq \epsilon$ for arbitrary $\epsilon > 0$. The methodology reshapes the unknown product $C = AB$ into a vector $c = \mathrm{vec}(C)$, constructs a low-rank operator $V$ encoding matrix–vector products, and solves the perturbed normal equations $(V^TV + \delta I)c = V^T u$ (where $u = Cv = A(Bv)$ for a test vector $v$).

The algorithm employs iterative linear solvers (steepest descent or conjugate gradient) with tailored perturbation $\delta = \Theta(\epsilon/n^2)$, reducing the effective condition number and iteration count via block-diagonalization and Sherman–Morrison–Woodbury identity. The total complexity is $\mathcal{O}(n^2 \log(1/\epsilon))$—the first deterministic scheme guaranteeing absolute Frobenius-norm error $\leq\epsilon$ in near-quadratic time. The same reduction generalizes to bilinear products beyond matrix multiplication, including convolutions and tensor contractions.

2. Neural Network Constructions for Multiplicative Functions

Feedforward neural architectures yield explicit constructive approximations to multiplication and related operations. In particular, with GELU activation, a two-layer network $\Phi_{\text{mul}}$ attains $$\|\Phi_{\text{mul}} - xy\|_{W^{m,\infty}([-M, M]^2)} \leq \varepsilon M^3$$ for all derivatives up to order $m$ (Yakovlev et al., 25 Dec 2025). The construction employs a finite-difference scheme on the GELU's Taylor expansion for squaring, then uses polarization ($xy = \frac{1}{4}((x+y)^2 - (x-y)^2)$) to build the bilinear block.

Every network parameter (depth, width, sparsity, weight bound) is given explicitly, with global derivative control. This $\Phi_{\text{mul}}$ block serves recursively for multi-argument products, division via partition-of-unity, and exponentiation through repeated multiplication, preserving uniform error and boundedness of all derivatives.

A distinct indicator-neuron architecture (01Neuro) (Chi, 7 Jul 2025) realizes constructive universal approximation for interaction terms $x_1 x_2$. Piecewise-constant grid partitioning and boosting over neuron ensembles achieve arbitrarily small $L_2$ error in the infinite-sample regime, with explicit depth, sparsity, and grid-resolution control. Empirical boosting and bagging further stabilize finite-sample behavior.

3. Bilinear and Fast Polynomial Algorithms for Structured Multiplication

Approximate bilinear algorithms, such as Smirnov's length-46 decomposition for $4\times4$ matrices (Smirnov, 2014), construct explicit sets of linear forms, scalar products, and reconstruction coefficients to realize $AB \approx C'$ for block-wise matrix multiplication. Sparse coefficients ($\pm1$) and the use of low-degree (order-3) polynomial error terms control both the arithmetic overhead and the numerical stability under recursion. When utilized as the base of divide-and-conquer recursions, such schemes lower the true exponent of multiplication below 3, with polynomial order ensuring bounded error propagation per level.

For integer multiplication, the Schönhage-Strassen algorithm and its interval-arithmetic extensions (Steinke et al., 2010) use FFT-based convolution, rounding, and rigorous interval error bounds to attain $O(n \log n)$ complexity and provably correct outputs, providing automatic detection of precision shortfall.

4. Operator-Theoretic and Matrix Projection Constructions

Approximation of multiplication operators—on $L^2$ spaces weighted by $w(x)$—via finite matrix projections underpins modern quadrature and numerical integration schemes (Sarmavuori et al., 2017, Sarmavuori et al., 2019). Projection onto orthonormal polynomial bases yields Jacobi matrices whose eigenvalues (nodes) and first-row coefficients (weights) reconstruct integrals $\int_\Omega f(x)w(x)\,dx$ as $e_1^T f(M_n) e_1$. Under spectral convergence theorems and operator inequalities (strong resolvent convergence, Jensen's), bounded, continuous, Riemann–Stieltjes integrable, and operator-convex functions $f$ admit convergent matrix-based approximations even for unbounded domains and improper integrals.

These constructions are inherently constructive: every step (basis selection, moment computation, eigendecomposition, evaluation) is algorithmic; error bounds and convergence are stated in operator norm, strong operator, and quadratic form error, with precise growth, monotonicity, and interlacing properties for node placement.

5. FFT, Convolutions, and Low-Rank Approximations for Large-Scale Multiplication

Efficient multiplication of large dense matrices leverages truncated decompositions and FFT-based algorithms. Truncated SVD, circulant decomposition, and sparse–dense residue splits enable fast approximate multiplication at $O(n^2 \log n)$ cost for $n\times n$ matrices, achieving relative Frobenius error $\approx 1\%$ (Kar et al., 27 Apr 2025). Selection among SVD, circulant, or FFT-based methods is guided by a priori error estimates, structure (periodicity, smoothness), and computational budget.

Recent algorithmic frameworks use CKSU polynomial embeddings and FFT-based convolutions (Pratt et al., 25 Oct 2025), exhibiting subcubic combinatorial multiplication ($O(n^{2.89})$) and, for approximate multiplication, Fourier-truncation and sketching techniques that deliver $O(r n^2)$ complexity and normalized error $O(r^{-1.1})$—strictly sublinear in the sketch size, outperforming classical $1/r$ tradeoffs. These methods prove especially effective for random and product-distributed matrices and scale to low-rank LLM training and inference settings.

6. Error Analysis, Stability, and Practical Considerations

Constructive approximation schemes provide explicit error bounds:

• For deterministic iterative systems, Frobenius-norm error is tightly linked to condition number and perturbation $\delta$, with stability traded against iteration complexity (Manne et al., 2014).
• Neural network blocks attain $W^{m,\infty}$ error control, ensuring all derivatives are bounded, critical for higher-order compositional approximators (Yakovlev et al., 25 Dec 2025).
• Bilinear and polynomial schemes quantify the propagation of amplitude and rounding error, with higher-order terms yielding better recursive stability at cost of sparse overhead (Smirnov, 2014, Steinke et al., 2010).
• Operator-theoretic projections guarantee convergence (strong, monotone, resolvent) across broad classes of functions, including those with polynomial, exponential, or singular growth (Sarmavuori et al., 2019).

Computational cost benchmarks are rigorously delineated: deterministic quadratic-time, $O(n^2 \log n)$ for truncated decompositions, quasi-linear for FFT or interval-based methods, and explicit scaling laws for depth/width in neural architectures, block size in binomial algorithms, and sketch dimension in sketch-based matmul.

7. Extensions and Generalizations

Constructive approximation of multiplication is extensible to:

• Bilinear maps beyond matrix multiplication: convolution, polynomial multiplication, tensor contractions (Manne et al., 2014, Yakovlev et al., 25 Dec 2025).
• Division, exponentiation, and higher-arity multiplication blocks in neural architectures, built modularly upon two-input product constructors (Yakovlev et al., 25 Dec 2025).
• Quadrature and operator calculus on function spaces, migratable to multivariate and distributional settings (Sarmavuori et al., 2017, Sarmavuori et al., 2019).
• Adaptive hybrid strategies, selecting decompositions a priori based on truncation error, circulant energy, or Fourier sparsity (Kar et al., 27 Apr 2025).

Empirical studies and theory jointly guide the selection of approximator, parameter regime, and algorithmic tradeoffs, with continuous advances in combinatorial, spectral, neural, and FFT-based constructions underpinning practical large-scale multiplication in both dense and structured domains.