Constrained Weiszfeld-type Algorithms

• The method generalizes the classical Weiszfeld algorithm by integrating projection and stabilization to handle explicit constraints and nondifferentiability.
• These algorithms guarantee monotonic descent and convergence to unique minimizers for nonsmooth convex problems defined on convex sets and manifolds.
• Applications in robust location analysis and robust PCA demonstrate practical performance with maintained feasibility and improved stability.

Constrained Weiszfeld-type algorithms constitute a class of iterative fixed-point methods for solving nonsmooth convex minimization problems involving the sum of Euclidean distances to a fixed collection of “anchor points,” while incorporating explicit constraints such as convex feasibility or manifold structure. These algorithms generalize the classical Weiszfeld method, originally developed for the unconstrained Fermat–Weber location problem, by integrating projections and stabilization mechanisms to ensure global descent, robustness at nondifferentiable configurations, and applicability to constrained settings and manifold optimization. Such methodologies are prominent in robust location analysis, low-rank data recovery, and optimization on convex or non-Euclidean domains.

1. Mathematical Formulation and Core Problems

The prototypical Weiszfeld-type problem targets minimization of a weighted sum of Euclidean (or other) distances:

$$f(x) = \sum_{i=1}^m w_i \|x - a_i\|, \quad x \in \mathbb{R}^n,$$

where $a_i$ are anchor points and $w_i > 0$ are weights. In the constrained setting, optimization is subject to additional requirements, e.g., $x \in C$ for a closed convex $C \subset \mathbb{R}^n$ or a nonlinear constraint such as $\|w\|=1$ (unit sphere). Notable instances include:

• The constrained Fermat–Weber problem: $\min_{x \in C} \sum_i w_i \|x-a_i\|$ (Nguyen, 2018, Torres, 2012)
• Robust principal component analysis (PCA): $$\min_{w \in \mathbb{R}^d,\,\|w\|=1} \sum_{i=1}^N \|P_{w^\perp}(x_i)\|_2$$, where $P_{w^\perp}$ is the orthogonal projector onto the hyperplane orthogonal to $w$ (Neumayer et al., 2019).

Convexity, coercivity, and strict convexity properties depend on the geometric configuration of anchor points and constraint sets. For closed convex sets and noncollinear anchors, strict convexity typically ensures uniqueness and stability of solutions (Nguyen, 2018, Torres, 2012).

2. Classical, Projected, and Stabilized Weiszfeld-type Iterations

The unconstrained Weiszfeld iteration is given by:

$$T(x) = \frac{\sum_{i=1}^m w_i a_i / \|x - a_i\|}{\sum_{i=1}^m w_i / \|x - a_i\|},$$

which performs a nonlinear fixed-point mapping toward the minimizer of $f(x)$. In the constrained setting, iterates must maintain feasibility, motivating the projected Weiszfeld iteration:

$x^{k+1} = P_C(T(x^k)),$

where $P_C$ denotes the metric projection onto $C$ (Nguyen, 2018, Torres, 2012).

For nonsmooth objectives or boundary cases (e.g., $x^k = a_j$), the classical mapping becomes undefined or singular. Extensions such as the Vardi–Zhang modification (Torres, 2012) and stabilization procedures navigate such points by convex combinations or by “stepping back” along specified segments to retain feasibility and ensure well-posedness at vertices.

On Riemannian manifolds such as spheres (e.g., in robust PCA), the iteration becomes:

$$w^{(r+1)} = \frac{C_{w^{(r)}} w^{(r)}}{\|C_{w^{(r)}} w^{(r)}\|},$$

with $C_w$ the weighted scatter matrix; an anchor-stabilized step is taken whenever iterates approach nondifferentiable configurations characterized by alignment with data vectors (Neumayer et al., 2019).

3. Convergence Properties and Descent Analysis

Constrained Weiszfeld-type methods are proved to have several key properties under structural assumptions:

• Monotonic Descent: Each iteration (projection or stabilization) strictly decreases the objective unless at a critical point (Neumayer et al., 2019, Nguyen, 2018, Torres, 2012).
• Fejér Monotonicity: Iterates are Fejér monotone with respect to the optimal solution, ensuring progress toward minimizers in Hilbert space formulations (Nguyen, 2018).
• Convergence Guarantee: For strictly convex $f$ and convex $C$, and avoiding pathological cases (iterates at anchors), sequences converge strongly to the unique global minimizer (Nguyen, 2018, Torres, 2012).
• Handling Nondifferentiable Points: At “anchor” points where the mapping is only one-sided differentiable, descent is re-established through stabilization (e.g., modified updates or segment interpolation) (Neumayer et al., 2019, Torres, 2012).

In manifold optimization (e.g., robust PCA), the Kurdyka–Łojasiewicz property of the objective and global convergence theorems (e.g., Attouch–Bolte–Svaiter framework) are invoked to ensure convergence to a single critical point, with vanishing step sizes as iterations proceed (Neumayer et al., 2019).

4. Algorithmic Variants and Practical Implementations

Several algorithmic forms arise depending on the constraint structure and application:

Algorithmic Variant	Key Step	Reference
Projected Weiszfeld	$x^{k+1} = P_C(T(x^k))$	(Nguyen, 2018)
Vardi–Zhang Stabilization	Blends standard Weiszfeld with safeguard at vertices	(Torres, 2012)
Anchor-stabilized Sphere Step	Picard-type + tailored update at anchor points on $S^{d-1}$	(Neumayer et al., 2019)
Segment projection for vertices	$\lambda$-based line search to remain feasible	(Torres, 2012)

Guarantees for each variant include feasibility, fixed-point optimality (solutions satisfy KKT conditions under regularity), and strict descent except at limit points. The per-iteration complexity is typically dominated by matrix–vector products and the cost of projection, ranging from $O(md)$ in location problems to $O(Nd^2)$ in robust PCA, except where projection is complex or not tractable in closed form (Neumayer et al., 2019, Nguyen, 2018, Torres, 2012).

5. Theoretical Properties: Stability, Uniqueness, and Extensions

Analytical results include:

• Existence and Uniqueness: Strict convexity of the weighted distance function, given noncollinearity and convex $C$, assures a unique minimizer (Nguyen, 2018, Torres, 2012).
• Stability: The minimizer and minimum value are continuous with respect to perturbations of anchor locations; the mapping from anchors to optimizer is locally Lipschitz (Nguyen, 2018).
• Generalization to Hilbert Spaces: All key results extend to real Hilbert spaces (including infinite dimensions), not just finite-dimensional Euclidean spaces (Nguyen, 2018).
• KKT Conditions: Fixed points of the constrained Weiszfeld map satisfy KKT optimality under differentiability and regularity (constraint qualification) (Torres, 2012).

Extensions to other norms (e.g., $l_p$), more general objective functions incorporating barriers or repulsion, and disconnected feasible sets are possible, provided the projection remains tractable (Torres, 2012).

6. Applications, Practical Performance, and Limitations

Constrained Weiszfeld-type algorithms are used both in continuous location optimization (facility location, network design) and in robust data analysis:

• In robust PCA, the algorithm leveraging Weiszfeld-style updates achieves robustness to outliers and empirical superiority to $\ell_1$-PCA and trimmed-PCA, requiring no tuning parameters and exhibiting fast local convergence (Neumayer et al., 2019).
• In constrained location problems, numerical experiments in two dimensions show global convergence, superior or equal solution quality compared to general nonlinear solvers (e.g., MATLAB’s fmincon), and always feasible iterates for arbitrary convex constraints (Torres, 2012).
• Main limitations arise from the cost of projection steps (especially for nontrivial $C$ or $\Omega$), and potentially degraded local convergence rates near the boundary of the feasible set (Nguyen, 2018, Torres, 2012).

7. Comparative Perspective and Distinctive Features

Relative to classical unconstrained Weiszfeld iterations, constrained and stabilized algorithms provide:

• Feasibility maintenance at every step for arbitrary closed convex sets via nonexpansive projection (Nguyen, 2018, Torres, 2012).
• Robustness to singularities at anchor points by explicit stabilization measures (Neumayer et al., 2019, Torres, 2012).
• Simple implementation in the basic case, with no requirement for subgradient or Hessian information—only function values and distances required (Torres, 2012).
• Explicitly proven convergence and optimality properties under mild hypotheses (strict convexity, closed convex constraint, regularity).

A plausible implication is that, for a broad class of nonsmooth constrained convex optimization problems characterized by distance-like objective terms, Weiszfeld-type projection-and-stabilization algorithms offer an efficient, theoretically guaranteed pathway with minimal requirements on objective or constraint smoothness.

References:

• "On the Robust PCA and Weiszfeld’s Algorithm" (Neumayer et al., 2019)
• "Constrained Fermat-Torricelli-Weber Problem in real Hilbert Spaces" (Nguyen, 2018)
• "A Weiszfeld-like algorithm for a Weber location problem constrained to a closed and convex set" (Torres, 2012)