Matrix-Vector Multiplication and Reduction

• MVMR is a computational paradigm that combines classical and generalized matrix-vector operations, enabling efficient data reduction and versatile analyses.
• It leverages semiring formulations and structured operator generalizations to support dynamic programming, spectral methods, and graph-based algorithms.
• Applications span scalable parallel processing, distributed computing, and advanced hardware implementations including in-memory, optical, and quantum architectures.

Matrix-Vector Multiplication and Reduction (MVMR), encompassing both classical and generalized matrix-vector operations, underpins a vast array of computational, scientific, and engineering domains. This concept unifies the processes of multiplying a matrix (or structured linear operator) by a vector and reducing, accumulating, or combining the partial results according to the algebraic structure of the underlying problem. Recent advances span theory (fine-grained lower/upper bounds), architectures (in-memory, optical, quantum, and compressed), parallel and distributed strategies, and application-driven formulations. The sections below provide a comprehensive technical survey.

1. Core Definitions and Algebraic Formulation

Matrix-Vector Multiplication and Reduction encompasses computations of the form

$$y = Mx, \quad \text{or} \quad y_i = \bigoplus_{j} (M_{ij} \otimes x_j)$$

where $M$ is a matrix (not necessarily dense or real-valued), $x$ is a vector, $\otimes$ and $\oplus$ are binary operations (potentially not standard multiplication and addition), and $y$ is the reduced result, possibly followed by an assign or post-processing step. This generalized semiring approach, as exemplified by the GIM-V model, accommodates classic algebraic (real/complex), Boolean, and min-plus systems, capturing PageRank, shortest paths, and other graph algorithms (Park et al., 2017).

Key properties of $(S, \otimes, \oplus)$ must include associativity, with distributivity and commutativity as warranted, enabling a broad class of generalized MVMR workloads, including dynamic programming, spectral methods, and iterative updates.

2. Computational Paradigms and Algorithmic Frameworks

Online and Fine-Grained Complexity

The Online Boolean Matrix-Vector Multiplication (OMV) paradigm asks for the sequential processing of a stream of query vectors $v_1, \ldots, v_t$ with a pre-fixed $n\times n$ Boolean matrix $M$, computing $Mv_i$ (in the Boolean semiring) before $v_{i+1}$ is seen. Classical combinatorial OMV algorithms achieved $O(n^3/\log^2 n)$ total time, with the OMV conjecture positing no $O(n^{3-\varepsilon})$ time randomized algorithm for any $\varepsilon>0$ (Larsen et al., 2016). This conjecture formed the basis of conditional barriers in dynamic and data-structure query lower bounds.

The breakthrough of (Larsen et al., 2016) leverages reductions from online vector-matrix-vector queries to the orthogonal vectors (OV) problem, which in turn are solved via explicit small matrix–matrix multiplications, yielding randomized OMV in $n^3/2^{\Omega(\sqrt{\log n})}$ total time and amortized $n^2/2^{\Omega(\sqrt{\log n})}$ per query after initial rounds. A further cell-probe construction achieves $O(n^{7/4}/\sqrt{w})$ probes which eliminates purely information-theoretic lower bound approaches.

Structured Matrices and Practical Complexity Gaps

Theoretical $\Omega(n^2)$ lower bounds are circumvented in practice for highly structured matrices. (Anand et al., 28 Feb 2025) shows that when the matrix $M$ over $\{0,1\}^{n\times n}$ has VC-dimension $d$, the query time for $Mv$ can be reduced to $\tilde O(n^{2-1/d})$ after $\tilde O(n^2)$ preprocessing, even with adversarial corruption in a subquadratic number of entries. The algorithm exploits spanning-tree differential compression (“mailman” method) in Hamming space, and Welzl-type crossing number bounds on VC-dimension, explaining the empirical success of fast MVMR routines for structured real-world datasets.

Distributed, Parallel, and Lossless Compressed Methods

For unstructured extraordinarily large systems, scalable strategies are essential:

• Nonzero-partitioned approach (partitioning by nonzeros, not rows/columns) maintains perfect flop balance across processors and robust communication characteristics, systematically constructing “overlap zones” to handle shared vector entries and ensuring only $O(\log P)$ additional communicator setup with no global decompositions required (Eckstein et al., 2018).
• Black-box kernel MVMR for fast kernel summation leverages hierarchical low-rank expansions (e.g., FMM) to reduce computational complexity from $O(N^2)$ to $O(N)$ for translation-invariant, non-oscillatory kernels, with OpenMP-parallelism and only a single vector reduction phase required (Wang et al., 2019).

Lossless grammar-compressed MVMR operates directly on a compressed representation $(R,C,V)$ of $M$, matching space and time to the $k$-th order entropy $H_k$ of the matrix—enabling space and run-time proportional to the compressed form and outperforming general-purpose compressors such as xz/gzip, and compressed linear algebra (CLA) libraries (Ferragina et al., 2022).

3. Hardware Architectures and Specialized Implementations

Multiplier-Free and In-Memory MVMR

Distributed Arithmetic (DA) replaces traditional MAC-based matrix-vector multipliers with LUT- and shift-add architectures, particularly effective for constant-weight matrices in in-memory (ReRAM) fabrics (Zeller et al., 2 Oct 2025). The DA scheme eliminates power-hungry ADCs and multipliers, trading LUT and peripheral circuitry for significantly lower area and energy, achieving $4.5\times$ lower latency and $12\times$ lower energy than bit-sliced in-memory VMM. The organization employs chunking high-dimensional vectors into $K$-bit groups for tractable LUT size.

Complex Arithmetic Optimization

For FPGA/ASICs and DSP pipelines, Winograd-based inner-product restructuring combined with Gauss's three-multiplier complex multiplication enables constant complex matrix–vector products with only $3N(M+1)/2$ real multipliers and $3M(N+2)+1.5N+2$ two-input real adders for $M\times N$ matrices, a significant reduction from the naïve $4MN$ multipliers (Cariow et al., 2014). The pipelined and block-structured algorithm supports high-throughput implementations for communication and signal processing.

Polynomial and Cryptographic Domains

In the context of post-quantum cryptographic schemes (e.g., Kyber), KyberMat splits each input polynomial into polyphase (even/odd) components, applies NTTs, and exploits sub-structure sharing to reduce the number of modular multiplications and additions in the matrix–vector stage by $12.5\%$–$25\%$. The hardware pipeline arranges all operations in a feed-forward manner, with no intermediate buffering, yielding a $90\%$ reduction in execution time and $66\times$ improvement in throughput performance on FPGAs compared to prior two-parallel implementations (Tan et al., 2023).

Coherent Optical and Quantum Reductions

Coherent free-space optical MVMR leverages cascaded SLMs, 4f imaging, and the Fourier-transform property of cylindrical lenses to compute $r_j = \sum_i M_{ji} v_i$ in parallel for up to $N=56$ at high throughput and low energy, with systematic pixel-by-pixel calibration. This architecture supports real-valued dense operations and is pertinent for optical neural networks and Ising machine acceleration (Spall et al., 2020).

Quantum algorithms for MVM attain worst-case to average-case reductions with quadratic overhead in the average-case success probability $\alpha$ (i.e., $O(\alpha^{-2} \log^2(1/\alpha))$), significantly improving upon previous quasi-polynomial constructions. The reduction exploits self-amplification/direct-product techniques without reliance on heavy analytic combinatorics, and rigorously composes block-wise reductions and verification circuits (Aggarwal et al., 17 Oct 2025).

4. Communication, Reduction, and Scalability Strategies

MVMR faces bottlenecks in I/O, communication, and data movement beyond computational arithmetic:

• PMV (pre-partitioned generalized matrix-vector multiplication) partitions $M$ once into $b\times b$ blocks, then selects at each iteration among horizontal, vertical, or hybrid placement to trade off between broadcasting $x$ vs. shuffling partial results $y$, with cost models capturing the impact of density, degree-thresholding, and placement strategies (Park et al., 2017).
• I/O-optimal strategies reduce shuffling and synchronization, favoring in-memory or network-bandwidth-limited systems in web- and graph-scale mining.
• Column reordering via similarity-score-driven heuristics (e.g., PathCover, Lin–Kernighan) can drive 16% further memory reduction and up to 25% speedup for compressed MVMR (Ferragina et al., 2022).

In distributed settings, “overlapped” vector representations enable pressure-free balancing, with sum-reductions (e.g., MPI_Allreduce) only over the reduced dimension, and with judicious partitioning the dominant communication cost stays $O(\log P)$ or less (Eckstein et al., 2018).

5. Theoretical Barriers, Open Questions, and Impact

• The OMV conjecture, long believed to undergird hardness for dynamic/query problems, can be shattered in the randomized and cell-probe models but remains an open landscape for deterministic and more general semiring cases (Larsen et al., 2016).
• Structure (manifest in VC-dimension or entropic compressibility) is a principal enabler of subquadratic algorithms, reconciling the dichotomy observed between theoretical lower bounds and practical performance (Anand et al., 28 Feb 2025).
• Black-box kernel MVMR has enabled scalable principal component analysis and Gaussian process inference for geostatistics, with $O(N)$ scaling; practical implementations achieve 19$\times$ parallel speedup on commodity multicore platforms (Wang et al., 2019).

Extensions to quantum fine-grained reduction frameworks, high-throughput analog/optical solutions, and application-tuned hardware demonstrate the cross-cutting centrality and evolution of MVMR as both a conceptual and technological primitive.

6. Comparative Hardware and Algorithmic Complexity Overview

Approach/Domain	Key Feature	Core Complexity/Scaling
Classical OMV	Online, Boolean, worst-case barrier	$O(n^3/2^{\Omega(\sqrt{\log n})})$ (Larsen et al., 2016)
VC-dim-structured	Preprocessing + subquadratic query	$\tilde O(n^2)$ + $\tilde O(n^{2-1/d})$ (Anand et al., 28 Feb 2025)
In-memory DA ReRAM	Multiplier-free, LUTs	$4.5\times$ latency, $12\times$ energy over bit-slice (Zeller et al., 2 Oct 2025)
Hardware/compressed (FPGA)	$O(MN)$ multipliers/adders	$3N(M+1)/2$ multipliers (Cariow et al., 2014)
Kernel black-box	Hierarchical $O(N)$	$O(N)$ (Wang et al., 2019)
PMV graph-mining	Pre-partition, hybrid comm.	Network I/O $\sim O(bn)$ per iter (Park et al., 2017)
Optical Engine	SLM + Fourier	$N=56$, $10^{14}$ ops/J (Spall et al., 2020)
Quantum reduction	Worst-case↔avg-case	$O(\alpha^{-2}\,\log^2(1/\alpha))$ (Aggarwal et al., 17 Oct 2025)
Grammar-compressed	Time/space ∼ $N H_k$	Direct $O(g)$, memory/proportional to entropy (Ferragina et al., 2022)

7. Applications and Future Directions

MVMR methods are foundational for dynamic graph algorithms (dynamic Laplacian solvers, effective resistance, triangle detection—now admitting subquadratic time for structured inputs (Anand et al., 28 Feb 2025)), scalable learning (kernel PCA, deep neural inference (Zeller et al., 2 Oct 2025, Spall et al., 2020)), cryptography (lattice-based PQC engines (Tan et al., 2023)), and numerical computing on sparse and dense regimes.

Future directions include: tightening bounds for structured but adversarially perturbed inputs; extending efficient in-memory, optical, and quantum MVMR to more general operator classes; and further integrating communication and I/O models into core algorithmic design. The interplay of algebraic, architectural, and information-theoretic insights will likely continue to advance the state of the art in matrix-vector multiplication and reduction.