Weak Orthogonal Matching Pursuit (WOMP)

• WOMP is a variant of Orthogonal Matching Pursuit that employs a weakness parameter to select atoms, balancing computational efficiency and robust recovery in compressive sensing.
• It utilizes global 2-coherence to bridge mutual coherence and RIP, providing tighter and more flexible recovery guarantees under both noisy and noiseless conditions.
• Thresholded adaptations like OMPT reduce computational complexity while maintaining comparable recovery performance to classical OMP in large-scale and practical settings.

Weak Orthogonal Matching Pursuit (WOMP) is a class of greedy algorithms for sparse signal recovery in the compressive sensing framework. WOMP generalizes classical Orthogonal Matching Pursuit (OMP) by relaxing the requirement that, at each iteration, the atom with maximal correlation is selected. Instead, WOMP admits atom selection based on a "weakness" parameter or threshold, enabling computational efficiency without compromising theoretical recovery guarantees under certain coherence or Restricted Isometry Property (RIP) conditions. Central to the analysis of WOMP is the introduction of the global 2-coherence, a measure unifying mutual coherence and RIP constants, facilitating tighter and more general recovery guarantees (Yang et al., 2014, Yang et al., 2013).

1. Algorithmic Framework

WOMP operates on a measurement model $f = \Phi a + w$, where $\Phi \in \mathbb{R}^{n \times d}$ is a dictionary with unit $\ell_2$-norm columns, $a \in \mathbb{R}^d$ is $k$-sparse ($\|a\|_0 \leq k$), and $w$ is bounded noise, $\|w\|_2 \leq \epsilon$. Key algorithmic variants are:

• WOMP with Weakness Parameter: At each iteration $s$, the algorithm selects any index $i$ satisfying

$$|\langle r_s, \phi_i \rangle| \geq \rho \max_{1 \leq j \leq d} |\langle r_s, \phi_j \rangle|$$

for a fixed $\rho \in (0, 1]$. $\rho=1$ recovers classical OMP, while lower $\rho$ values allow sub-optimal (but computationally cheaper) selections.

• Thresholded WOMP (OMPT): Rather than selecting a single atom per iteration, all atoms $i$ with

$|\langle \phi_i, r^k \rangle| \geq \tau \|r^k\|_2$

for some threshold $\tau \in (0,1)$, are incorporated. Support is updated by augmenting with the resulting indices, followed by a least-squares estimate restricted to the current support (Yang et al., 2013).

The iteration is terminated if the residual norm decreases below a preset threshold or if the presumed sparsity level is reached. The outputs are support and coefficients for the selected atoms.

2. Global 2-Coherence: Definition and Properties

Global 2-coherence generalizes mutual coherence for finer quantification of dictionary redundancy. For integer $k \geq 1$, it is defined as:

$$\nu_k(\Phi) := \max_{i \in [d]} \max_{\substack{\Lambda \subseteq [d] \setminus \{i\} \ |\Lambda| \leq k}} \left( \sum_{j \in \Lambda} \langle \phi_i, \phi_j \rangle^2 \right)^{1/2}$$

This measures the maximal collective correlation (in $\ell_2$) any atom $\phi_i$ has with any subset of $k$ other dictionary atoms. Mutual coherence is the special case $$\nu_1(\Phi) = M(\Phi) = \max_{i \neq j} |\langle \phi_i, \phi_j \rangle|$$.

Thresholded versions use a closely related index $\mu_{2,t}(\Phi)$, defined analogously (Yang et al., 2013). These indices provide an explicit bridge between mutual coherence and RIP constants.

3. Relations to Mutual Coherence and RIP Constants

Global 2-coherence is tightly connected to mutual coherence $M$ and the RIP constant $\delta_k$:

$$M \leq \nu_{k-1}(\Phi) \leq \delta_k \leq \sqrt{k-1}\;\nu_{k-1}(\Phi) \leq (k-1)M$$

This relationship elucidates how global 2-coherence interpolates between per-pair (mutual) and collective (RIP) measures of dictionary redundancy, enabling new analytic techniques for establishing recovery guarantees (Yang et al., 2014, Yang et al., 2013).

4. Sparse Recovery Guarantees

Noisy Case: Suppose $a_{min} := \min_{j \in \mathrm{supp}(a)} |a_j|$. WOMP with weakness parameter $\rho$ and stopping at $k$ iterations recovers the correct support and achieves

$$\|\hat a - a\|_2 \leq \frac{\epsilon}{\sqrt{1 - \delta_k}}$$

provided both

$\sqrt{k} \nu_k(\Phi) < \rho (1 - \delta_k)$

and

$$\epsilon < \frac{\rho (1 - \delta_k) - \sqrt{k} \nu_k(\Phi)}{1 + \rho} a_{min}$$

(Yang et al., 2014).

Noiseless Case, $\rho=1$ (OMP): Exact $k$-term support recovery is guaranteed under the improved RIP condition

$\delta_k + \sqrt{k}\, \delta_{k+1} < 1$

which strictly improves upon the classical $\delta_{k+1} < 1/(1+\sqrt{k})$ guarantee (Yang et al., 2014).

Thresholded OMPT: For thresholded selection, support recovery occurs in $s$ iterations, provided

$\delta_s + \sqrt{s}\,\mu_{2,s} < 1$

and for threshold parameter $$\tau \in (\tfrac{\mu_{2,s}}{\sqrt{1-\delta_s}},\,\tfrac{\sqrt{1-\delta_s}}{\sqrt{s}}]$$ (Yang et al., 2013). In mutual coherence terms, the traditional bound $M < 1/(2s-1)$ is recovered with appropriate $\tau$.

Recovery of the support proceeds by induction: at each iteration, projected noise plus off-support correlations (quantified by global 2-coherence and RIP) is controlled, ensuring correct support identification. Proofs rely on bounding residual correlations, and invoke the collective structure captured by global 2-coherence.

5. Computational Complexity

WOMP/OMPT algorithms achieve substantial complexity reduction relative to classical OMP. Standard OMP requires $O(s n d)$ inner products and several $O(s^3)$ least-squares solves. For thresholded variants (OMPT), only those columns passing the threshold $\tau$ need correlation computation, and the average number of inner products per iteration remains essentially constant with respect to iteration count and sparsity level, often much less than OMP’s linear scaling in $d$. Empirical studies with hybrid dictionaries (e.g., $$[\mathrm{I}, \mathrm{F}] \in \mathbb{R}^{128 \times 256}$$, $M=1/\sqrt{128}$) show that OMPT achieves nearly the same recovery probability as OMP up to moderate sparsity while requiring significantly fewer inner products (Yang et al., 2013).

6. Theoretical and Practical Considerations

Assumptions underlying WOMP analyses include normalized dictionary columns and exact sparsity (plus bounded noise). The sufficient recovery conditions derived using global 2-coherence are not known to be necessary; their sharpness remains an open question. WOMP closes part of the gap between coherence and RIP-based analyses, providing unifying and, in some regimes, improved guarantees for greedy methods.

No numerical simulations in (Yang et al., 2014) illustrate tightness of the RIP bounds for WOMP; however, (Yang et al., 2013) includes empirical evidence regarding the trade-off between complexity and performance in thresholded WOMP. A plausible implication is that for large dictionaries and practical compressive sensing setups, thresholded variants yield substantial speedup without compromising recovery capability under similar coherence regimes.

7. Summary and Impact

Weak Orthogonal Matching Pursuit, through its flexible atom selection and analysis via global 2-coherence, provides a framework for efficient and theoretically robust sparse recovery. By relating global 2-coherence tightly to mutual coherence and RIP, it enables more general, and in some parameters strictly better, conditions for exact recovery compared to classical OMP. These results establish WOMP (and OMPT) as practical alternatives to OMP, enjoying identical performance guarantees when dictionary coherence and RIP properties are favorable, but with reduced computational demand (Yang et al., 2014, Yang et al., 2013).