WGMV Primal-Dual Algorithm

• The WGMV primal–dual algorithm is an operator splitting method for composite convex optimization and saddle-point problems with weakly convex structure.
• It employs a modified duality gap and inf‐sharp error bounds to ensure local linear convergence even in nonconvex settings.
• Numerical experiments in large-scale ℓ1-regularization and image deblurring demonstrate its practical effectiveness and robustness.

The WGMV (Weakly convex–Gapped–Modified‐Variational) primal–dual algorithm is an operator splitting method for composite convex optimization and saddle-point problems in Hilbert spaces, targeting models with weakly convex (possibly nonsmooth) structure in the primal component. Unlike classical schemes, WGMV achieves local linear convergence rates under sharpness of a modified duality gap even when the objective is nonconvex. The methodology is rooted in recent advances on proximal subdifferentials, inf-sharp error bounds, and alternate definitions of duality gap, broadening the applicability of primal-dual hybrid gradient methods well beyond the standard convex–concave setting (Bednarczuk et al., 2024).

1. Problem Formulation and Mathematical Framework

WGMV operates in real Hilbert spaces $X$ and $Y$, equipped with standard inner products. The setting involves composite minimization: $\min_{x\in X} F(x) := f(x) + g(Lx)$ where:

• $f: X \to (-\infty,+\infty]$ is proper, lower-semicontinuous, and $\rho$-weakly convex,
• $g: Y \to (-\infty,+\infty]$ is proper, lower-semicontinuous, and convex,
• $L: X \to Y$ is a bounded linear operator.

The associated saddle-point (Lagrangian) problem is: $$\min_{x\in X} \max_{y\in Y} \mathcal{L}(x,y) = f(x) + \langle Lx, y \rangle - g^*(y)$$ where $g^*$ is the convex conjugate of $g$ (Bednarczuk et al., 2024).

2. Weak Convexity and Proximal Subgradients

A function $f$ is $\rho$-weakly convex if: $$f(\lambda x + (1-\lambda)y) \leq \lambda f(x) + (1-\lambda)f(y) + \frac{\rho}{2}\lambda(1-\lambda)\|x-y\|^2$$ for all $x,y \in X$, $\lambda \in [0,1]$. This is equivalent to $f + \frac{\rho}{2}\|\cdot\|^2$ being convex.

For weakly convex $f$, the global proximal subdifferential at $x_0$ is: $$\partial_\rho f(x_0) = \left\{v \in X \mid f(x) - f(x_0) \geq \langle v, x - x_0 \rangle - \frac{\rho}{2}\|x-x_0\|^2,\, \forall x \in X \right\}$$ This coincides with the Clarke subdifferential, guaranteeing nonemptiness and facilitating the subsequent algorithmic steps (Bednarczuk et al., 2024).

3. Modified Gap Function and Inf-Sharpness

In contrast to the standard duality gap, the WGMV algorithm leverages a modified gap function: $$\mathcal{H}(x, y) := \inf_{(x^*, y^*) \in S} \left\{ \mathcal{L}(x, y^*) - \mathcal{L}(x^*, y) \right\}$$ where $S$ is the set of saddle points. Inf-sharpness is defined by the existence of $\mu > 0$ such that: $$\mathcal{H}(x, y) \geq \mu\, \mathrm{dist}\big((x, y), S\big) \quad \forall (x, y) \in X \times Y$$ This gap vanishes exactly on $S$ and provides a local error-bound type property necessary for linear convergence analysis (Bednarczuk et al., 2024).

4. Algorithm Structure and Convergence Guarantees

The “dual-first” WGMV primal–dual iteration is: $$\begin{aligned} & y_{n+1} = \arg\min_{y \in Y} \left\{ g^*(y) + \frac{1}{2\tau}\|y - (y_n + \tau Lx_n)\|^2 \right\} \ & \bar{y}_{n+1} = y_{n+1} + \theta (y_{n+1} - y_n) \ & x_{n+1} = \arg\min_{x \in X} \left\{ f(x) + \frac{1}{2\sigma} \|x - (x_n - \sigma L^*\bar{y}_{n+1})\|^2 \right\} \end{aligned}$$ with parameters $\tau, \sigma > 0$, $\theta \in [0,1]$ subject to $\sigma \rho < 1$, $\sqrt{\sigma \tau}\|L\| < 1$, and the sharpness-related constraint $\sigma\rho + \theta \sqrt{\sigma \tau}\|L\| < 1$. The optimality conditions are written via proximal subgradients: $$\frac{y_n - y_{n+1}}{\tau} + Lx_n \in \partial g^*(y_{n+1}),\qquad \frac{x_n - x_{n+1}}{\sigma} - L^*\bar{y}_{n+1} \in \partial_\rho f(x_{n+1})$$ Under inf-sharpness, geometric (linear) convergence of the distance to the saddle set is obtained within a neighborhood of attraction (Bednarczuk et al., 2024). The radius of convergence $R$ depends on problem and step size parameters.

5. Relationship to Classical Convex-Concave Algorithms

In convex–concave scenarios, methods such as Chambolle–Pock exhibit at most sublinear $O(1/N)$ or $O(1/N^2)$ convergence for standard ergodic duality gaps. WGMV extends this to yield local linear convergence even when $f$ is only weakly convex and sharpness is satisfied. The sharpness of only the primal function $f+g\circ L$ suffices for local convergence of the primal iterates $x_n$; the rate (linear/sublinear) is governed by the decay speed of the dual residual $\mathrm{dist}(y_n, \partial g(Lx_{n+1}))$ (Bednarczuk et al., 2024).

6. Practical Aspects and Numerical Performance

Experiments detail the behavior in several settings:

• Synthetic scalar models with unique saddle points demonstrated linear convergence within the convergence radius, divergence otherwise.
• Large-scale $\ell_1$-regularization ($n=3000$, $m=2000$) compared the standard convex and weakly convex variants; the latter converged faster near the solution and exhibited greater noise robustness.
• Image deblurring and total-variation denoising on benchmarks (BSD68) indicated that the weakly convex formulation provided sharper reconstructions and higher PSNR, especially at moderate and high noise.

Closed-form proximal updates are available for many weakly convex penalties, e.g., $|\|x\|^2 - c|$. The constants governing sharpness and the convergence radius are not known a priori, so practical deployment relies on heuristic step size tuning and monitoring the decay of $\|x_{n+1} - x_n\|$ and $\|y_{n+1} - y_n\|$ (Bednarczuk et al., 2024).

7. Limitations and Directions for Further Research

The theory establishes only local convergence—initialization outside the identified basin of attraction may lead to convergence to extraneous critical points or divergence. Extensions include adaptive step size selection (e.g., via line search), block-coordinate or stochastic variants, and schemes for more general nonconvex–nonconcave saddle-point problems under two-block error-bound (sharpness) assumptions.

The WGMV primal–dual algorithm integrates proximal-splitting methodology with a modified duality gap and sharpness-based error bounds, extending the applicability of primal–dual approaches to weakly convex problems with provable local linear rates of convergence (Bednarczuk et al., 2024).