AMP: Approximate Message Passing

• AMP defines a framework of iterative algorithms that leverages Onsager corrections to decouple dependencies in high-dimensional inverse problems.
• Its state evolution formalism precisely predicts performance, enabling reliable and scalable recovery even in noisy, large-scale settings.
• Extensions such as OAMP, VAMP, and CAMP broaden AMP's applicability to complex measurement ensembles and distributed computing frameworks.

Approximate Message Passing (AMP) is a class of iterative algorithms central to modern high-dimensional signal processing, statistical inference, and machine learning. AMP variants provide low-complexity, scalable solutions to large linear and non-linear inverse problems, and under suitable conditions, their dynamics are precisely predicted by the so-called state evolution formalism. Beyond their algorithmic role, the AMP framework has broad conceptual reach—in optimization, coding theory, distributed computing, probabilistic graphical models, and statistical physics. This article surveys the foundational principles, mathematical structures, algorithmic innovations, and major applications of AMP and its extensions.

1. Origins and Core Mathematical Principles

AMP algorithms were initially motivated by the challenge of efficient sparse-signal recovery in underdetermined linear systems (compressed sensing), where classical approaches (Lasso, basis pursuit) may be computationally demanding or limited by model mismatch. The canonical AMP iteration, for observations $y = Ax + w$ (with $A \in \mathbb{R}^{M \times N}$, $M \ll N$), is:

• Residual update:

$$z_t = y - A x_t + \frac{z_{t-1}}{M} \mathrm{div}\{D_{t-1}(r_{t-1})\}, \quad z_{-1}:=0$$

• Pseudo-data:

$r_t = x_t + A^T z_t$

• Nonlinear (denoising) step:

$$x_{t+1} = D_t(r_t; \hat\sigma_t), \quad \hat\sigma_t^2 = \|z_t\|^2 / M$$

where $D_t$ is a (possibly nonlinear) denoiser, and $\mathrm{div}\{D_t(r)\}$ is the empirical divergence.

The Onsager term (the second term in the residual update) is pivotal: it cancels dependencies arising from iterative reuse of the measurement matrix $A$, enabling the effective decoupling of the system. For random sensing matrices with i.i.d. Gaussian entries, AMP iterates are asymptotically characterized by a scalar recursion—state evolution (SE)—making AMP highly analyzable and predictable (Xue et al., 2016).

2. State Evolution and Universality

A distinguishing feature of AMP-type algorithms is that their asymptotic behavior (e.g., mean squared error per iteration) is governed by the state evolution recursion. For the classical setting with $A$ i.i.d. Gaussian and the denoiser $A \in \mathbb{R}^{M \times N}$0 componentwise:

$A \in \mathbb{R}^{M \times N}$1

This dynamic can be rigorously justified—for polynomial and Lipschitz nonlinearities, and for matrices with sub-Gaussian entries—using combinatorial expansions (e.g., tree pairings and Wick algebra), with non-Gaussian corrections vanishing for $A \in \mathbb{R}^{M \times N}$2 ($A \in \mathbb{R}^{M \times N}$3 polynomial degree) (Kogan, 8 Dec 2025). State evolution "universality" implies that AMP reliably predicts its performance across a broad matrix ensemble, enabling its use far beyond the strict Gaussian regime.

3. Extensions: OAMP, VAMP, MAMP, and Rotationally-Invariant Models

While classical AMP is analytically tractable for i.i.d. Gaussian matrices, it can diverge or underperform for ill-conditioned, structured, or non-Gaussian measurement ensembles. Several powerful generalizations have been developed:

• Orthogonal AMP (OAMP) eliminates explicit computation of Onsager terms by imposing a divergence-free constraint on the denoiser, restoring the signal-plus-Gaussian-noise property in the pseudo-data and enabling applicability to right-orthogonally-invariant matrices. At iteration $A \in \mathbb{R}^{M \times N}$4:

$A \in \mathbb{R}^{M \times N}$5

where $A \in \mathbb{R}^{M \times N}$6 is a linear estimator (e.g., LMMSE). Nonlinear steps use $A \in \mathbb{R}^{M \times N}$7 with $A \in \mathbb{R}^{M \times N}$8 (Xue et al., 2016, Cheng et al., 2023).

• Vector AMP (VAMP) and Memory AMP (MAMP) extend OAMP by allowing long-memory in the estimation modules, further increasing stability for general right-unitarily-invariant matrices. In MAMP, memory modules depend on all previous estimates and residuals, subject to a strong set of orthogonality constraints. The Bayes-optimal MAMP (BO-MAMP) uses matched-filter based linear estimation and MMSE denoising, with state evolution shown to match the fixed point of high-complexity OAMP/VAMP (Liu et al., 2020, Liu et al., 2021).
• Rotationally-Invariant AMP (RI-AMP) provides a systematic construction of AMP algorithms for arbitrary rotationally-invariant ensembles, deriving Onsager-type corrections using free-probability theory (free cumulants of the spectral law), yielding a unification of previous OAMP/VAMP approaches (Liu et al., 2024). The framework easily delivers "divergence-free" and "matrix processing" variants, with rigorous state evolution for a wide model class.
• Convolutional AMP (CAMP) replaces the scalar Onsager term with a convolution over all past iterations, designed to preserve asymptotic Gaussianity of the error under orthogonally invariant ensembles. When SE converges to a unique fixed point, Bayes-optimal CAMP matches information-theoretic MMSE, at reduced computational complexity compared to OAMP/VAMP (Takeuchi, 2020, Takeuchi, 2021).

4. Practical Algorithms and Applications

AMP-type methods exhibit remarkable computational and estimation performance across various domains:

• Compressed sensing and sparse recovery: AMP—especially with advanced denoisers (e.g., BM3D, SURE-LET)—delivers state-of-the-art efficiency and accuracy, with provable SE-predicted MSE. Denoising-AMP (D-AMP) and divergence-free D-OAMP enable plug-in use of arbitrary denoisers, broadening the method's reach to real-world imaging (Xue et al., 2016).
• Coding and capacity: For large random matrix systems with coding constraints, AMP with properly matched decoding achieves the constrained capacity, with numerical gaps within $A \in \mathbb{R}^{M \times N}$9 dB of the Shannon limit for QPSK+LDPC codes, outperforming alternatives by several dB (Liu et al., 2021, Liu et al., 2019).
• Multi-processor distributed inference: Multi-Processor AMP (MP-AMP) variants support partitioning of $M \ll N$0 by rows or columns, allowing massive distributed inference with provable SE and guaranteed MMSE, and efficient lossy communication protocols (Zhu et al., 2017).
• Non-linear and invariant models: Generalized AMP (GAMP), Spectral AMP (S-AMP), and their non-linear extensions support quantized, clipped, or other non-linear systems, leveraging free-probability to adapt the Onsager term for invariant ensembles (Çakmak et al., 2015, Cheng et al., 2023).
• Bayesian and optimal estimation: When the denoiser matches the Bayes MMSE and the state evolution fixed point is unique, AMP, OAMP, CAMP, and MAMP all achieve Bayes-optimal performance for their respective matrix ensembles (Liu et al., 2020, Takeuchi, 2020, Liu et al., 2024).
• Overflow and numerical stability in large-scale systems: Memory AMP can be modified using overflow-avoiding transformations and complexity-reducing recursions without degradation of convergence or SE-predicted MSE (Huang et al., 2024).
• Specialized applications: Hard-thresholding AMP, usually unstable, succeeds in extreme-sparsity THz channel estimation, achieving near-oracle NMSE (Schram et al., 2019). Greedy adaptive matching pursuit ("AMP") efficiently solves spike-and-slab MAP sparse inference problems, outperforming convex relaxations (Vu et al., 2016). In graphical models, AMP chain graphs have induced a constraint-based learning algorithm with well-defined faithfulness assumptions but no Meek-type extension (Peña, 2012).

5. Impact in Distributed Systems, Blockchains, and Scientific Computing

AMP also denotes unrelated but significant frameworks in other domains:

• Model Parallelism: The "AMP" system for distributed ML enables automatic, heterogeneity-aware search for model parallel strategies across complex models and cluster architectures, achieving up to $M \ll N$1 higher throughput over prior heuristics (Li et al., 2022).
• Blockchain Protocols: The Arc Multi-Proposer Protocol ("AMP") atop Tendermint provides a robust, censorship-mitigating blockchain architecture, separating transaction dissemination from consensus, delivering "bounded inclusion guarantees", deterministic ordering, and enhanced throughput (Cason et al., 22 May 2026).
• Antimicrobial Peptide Design: MAC-AMP is a multi-agent reinforcement learning and peer-review platform for de novo antimicrobial peptide discovery, boasting superior antibacterial activity, structural reliability, and explainability compared to LLM, GAN, and diffusion baselines (Zhou et al., 16 Feb 2026).
• Asteroseismology: The Asteroseismic Modeling Portal ("AMP") exposes complex asteroseismic pipeline computations through a science-driven, web-based application; robust orchestration using Python/Django and scientific grid computing enables efficient astrophysical model optimization (Woitaszek et al., 2010).

6. Algorithmic Structures and State Evolution: Summary Table

The following table summarizes key AMP algorithm families and their principal analytic properties:

Algorithm Family	Matrix Ensemble	Onsager Structure	SE Validity	Bayes-optimality (when SE unique)
Classical AMP	i.i.d. Gaussian	Scalar divergence	Exact	Yes
OAMP / VAMP	Right-unitarily invariant	Divergence-free denoiser	Exact	Yes
CAMP	Orthogonally invariant	Convolutional Onsager	Exact (if converges)	Yes
MAMP / SS-MAMP	Right-unitarily invariant	Full-memory orthogonality	Exact	Yes
S-AMP	Rotationally invariant	R-transform based	Exact	Yes
RI-AMP (Unified)	General rotationally invariant	Free cumulant/trace-free	Exact	Yes

AMP-type algorithms thus provide a hierarchy of methods that interpolate between algorithmic simplicity and universality, with trade-offs in per-iteration complexity versus model generality.

7. Theoretical Guarantees, Limitations, and Open Problems

AMP methodologies benefit from rigorous non-asymptotic analysis in several regimes, most notably in the large-system limit for random matrix ensembles with well-understood spectra. However, limitations include:

• Divergence or slow convergence for ill-conditioned or adversarially structured $M \ll N$2 (classical AMP).
• Fixed-point multiplicity or lack of uniqueness in model regimes with phase transitions, leading to performance gaps unless proper initialization or code-matching is performed.
• For practical problems with finite $M \ll N$3, non-independent entries, or model mismatch, state evolution can mispredict real-world behavior; damping, memory extension, or empirical tuning may be beneficial (Liu et al., 2020, Liu et al., 2021, Huang et al., 2024).
• In constraint-based learning of AMP chain graphs, the lack of a Meek-type extension precludes efficient score+search algorithms under the composition assumption (Peña, 2012).

Continued research focuses on numerical stability in large-scale regimes, extensions to high-order tensor inference, integration with deep learning workflows, and principled design of distributed and privacy-preserving inference architectures.

---

References:

• "D-OAMP: A Denoising-based Signal Recovery Algorithm for Compressed Sensing" (Xue et al., 2016)
• "Capacity Optimality of AMP in Coded Systems" (Liu et al., 2021)
• "S-AMP for Non-linear Observation Models" (Çakmak et al., 2015)
• "AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness" (Li et al., 2022)
• "AMP: Arc Multi-Proposer Protocol with Bounded Inclusion Guarantees" (Cason et al., 22 May 2026)
• "Memory AMP" (Liu et al., 2020)
• "Bayes-Optimal Convolutional AMP" (Takeuchi, 2020)
• "Overflow-Avoiding Memory AMP" (Huang et al., 2024)
• "Orthogonal AMP for Problems with Multiple Measurement Vectors and/or Multiple Transforms" (Cheng et al., 2023)
• "Unifying AMP Algorithms for Rotationally-Invariant Models" (Liu et al., 2024)
• "Sufficient-Statistic Memory AMP" (Liu et al., 2021)
• "MAC-AMP: A Closed-Loop Multi-Agent Collaboration System for Multi-Objective Antimicrobial Peptide Design" (Zhou et al., 16 Feb 2026)
• "Approximate Message Passing for Indoor THz Channel Estimation" (Schram et al., 2019)
• "Adaptive matching pursuit for sparse signal recovery" (Vu et al., 2016)
• "Learning AMP Chain Graphs under Faithfulness" (Peña, 2012)
• "Universality of AMP via Tree Pairings" (Kogan, 8 Dec 2025)
• "AMP: A Science-driven Web-based Application for the TeraGrid" (Woitaszek et al., 2010)
• "An Overview of Multi-Processor Approximate Message Passing" (Zhu et al., 2017)
• "Bayes-Optimal Convolutional AMP" (Takeuchi, 2021)
• "Capacity Optimality of AMP in Coded Systems" (Liu et al., 2019)