Mordukhovich Stationarity in Sparsity Optimization

• The topic introduces cardinality-constrained Mordukhovich stationarity, which provides necessary optimality conditions using variational analysis for sparse, nonconvex problems.
• It details continuous relaxations and sequential stationarity conditions that overcome combinatorial challenges in ℓ0-constrained optimization.
• The framework bridges theory and algorithm design, offering practical methods for applications in compressed sensing, sparse learning, and signal processing.

Cardinality-constrained Mordukhovich stationarity points generalize classical first-order optimality concepts to problems with $\ell_0$-type constraints on the support size of solutions. The theory provides a normal-cone-based characterization of stationarity for sparsity-constrained optimization, grounded in variational analysis, bringing meaningful necessary conditions to settings where combinatorial constraints or lack of constraint qualification preclude application of Karush–Kuhn–Tucker (KKT) theory. This framework is crucial for both theoretical understanding and algorithmic development in high-dimensional, sparse, or structured optimization.

1. Problem Formulation and Continuous Relaxation

The canonical cardinality-constrained optimization problem (CCOP) is stated as

$$\min_{x\in\R^n}\; f(x) \quad \text{s.t. } g(x)\leq 0,\, h(x)=0,\; \|x\|_0 \leq k,$$

where $f:\R^n\to\R$, $g:\R^n\to\R^m$, $h:\R^n\to\R^p$ are continuously differentiable and the cardinality constraint $\|x\|_0 \leq k$ enforces sparsity. The nonconvex and discontinuous nature of the $\ell_0$ constraint necessitates alternative formulations for analysis.

A popular continuous relaxation introduces auxiliary variables $y\in\R^n$:

$$\min_{x,y} \;f(x) \quad\text{s.t.}\; g(x)\leq 0,\; h(x) = 0,\; x\circ y = 0,\; n - k - e^\top y \leq 0,\; y \leq e,$$

where $e$ is the all-ones vector and $x\circ y$ denotes the Hadamard product. This reformulation is equivalent to CCOP in terms of local/global minima, enabling variational analysis on a continuous feasible set (Pang et al., 2021).

2. Mordukhovich (Limiting Normal Cone) Stationarity

Cardinality-constrained Mordukhovich (CC-M) stationarity utilizes the limiting normal cone to characterize feasible directions at a point $(x^*,y^*)$ on the constraint set:

$$N_{\Omega}(x^*,y^*) := \limsup_{(x,y)\to(x^*,y^*)} \{v :\, \exists \alpha\geq 0,\; v \in \alpha N^P_{\Omega}(x,y)\},$$

where $N^P$ is the proximal normal cone.

A point $(x^*,y^*)$ is called CC-M stationary if

$$0 \in \nabla_x f(x^*) \times \{0\} + N_{\Omega}(x^*,y^*),$$

which, in multiplier form, admits $(\lambda, \mu, \gamma, \delta, \eta)$ such that

$\nabla f(x^*) + \nabla g(x^*) \lambda + \nabla h(x^*) \mu + \gamma = 0,\quad \lambda_i\geq 0,\; \lambda_i g_i(x^*)=0,\, \gamma_\imath=0~\forall \imath \in I_\pm(x^*),$

with appropriate complementarity and sign conditions reflecting the cardinality and auxiliary constraints (Pang et al., 2021).

For directly $\ell_0$-constrained sets $C_s$, the Mordukhovich normal cone is

$$N_{C_s}(x^*) = \{ \gamma\in\R^n:\gamma_{I_1(x^*)}=0 \}$$

with $I_1(x^*) = \{i:x^*_i\ne 0\}$, so CC-M stationarity for

$\min f(x)\;\text{s.t. } x\in C\cap C_s$

is

$0 \in \nabla f(x^*) + N_C(x^*) + N_{C_s}(x^*)$

(Mousavi et al., 18 Jan 2026, Xiao et al., 2022). This captures all directions blocked by the cardinality constraint.

3. Sequential and Approximate Stationarity Conditions

Classical stationarity conditions are not directly sequential; thus, CC-PAM-stationarity (Cardinality-Constrained Positive Approximate Mordukhovich stationarity) strengthens sequential optimality (Pang et al., 2021):

• $x^*$ is CC-PAM stationary if there exist sequences $(x^k,\lambda^k,\mu^k,\gamma^k)$ converging to $(x^*,\lambda^*,\mu^*,\gamma^*)$ satisfying
  • Stationarity (residual vanishes)
  • Multiplier zero conditions on inactive constraints
  • Positive approximation: if a multiplier component is asymptotically nonzero relative to the block maximum, then its corresponding product with the constraint's residual is strictly positive.

This strictly reduces the pool of candidates compared to earlier sequential notions (e.g., approximate Mordukhovich stationarity, CC-AM), providing a robust first-order necessary condition in combinatorial sparsity settings.

A tailored constraint qualification, CC-PAM-regularity, asserts that all approximate limiting normals collapse to the exact normal cone, ensuring that CC-PAM-sequential stationarity implies true CC-M stationarity (Pang et al., 2021).

4. Constraint Qualifications and Second-Order Theory

For Mordukhovich stationarity to characterize local minimizers, certain constraint qualifications must hold:

• RCPLD (Relaxed Constant Positive Linear Dependence) suffices for metric subregularity/error bounds, guaranteeing M-stationarity for local minimizers (Xiao et al., 2022).
• CC-LICQ (Cardinality-Constrained Linear Independence Constraint Qualification) is generically active, and under CC-LICQ, all M-stationary points are nondegenerate in a dense open subset of problem data (Shikhman et al., 2021).
• Second-order necessary and sufficient conditions under CC-LICQ yield uniqueness and strict local optimality for M-stationary points: positivity or strong positivity of Lagrangian Hessians on the critical cones ensures isolation and robustness (Bucher et al., 2017, Shikhman et al., 2021).

Genericity and stability results show that nondegeneracy and strong stability (in Kojima's sense) are generically satisfied at M-stationary points, and that Morse-theoretic structures are governed by the algebraic M-index/critical cone dimensions.

5. Algorithmic Implications and Convergence

Practical algorithms for cardinality-constrained optimization increasingly target CC-M-stationary points, acknowledging the nonconvex, combinatorial nature of the feasible sets:

• Augmented Lagrangian methods (SALM) generate CC-PAM sequences; feasible accumulation points are CC-PAM-stationary and, under CC-PAM-regularity, CC-M-stationary (Pang et al., 2021).
• Quasi-Newton penalty decomposition methods decompose the problem into smooth subproblems in $x$ and sparse projections in $y$, ensuring, under mild growth and regularization assumptions, global convergence to basic feasible and CC-M-stationary points (Mousavi et al., 18 Jan 2026).
• Scholtes-type regularizations and continuous reformulations guarantee sequence convergence to M-stationary points under suitable constraint qualifications and second-order conditions (Bucher et al., 2017).

The CC-M framework enables the development of efficient, robust algorithms that do not require enumeration of all support sets and provide verifiable stopping criteria via sequential stationarity tests.

6. Structural and Global Topological Aspects

Mordukhovich stationarity theory reveals structural properties of the landscape:

• The feasible set is a union of subspaces; tangent and normal cone formulas are explicit and depend on the current support $I(x)$ (Xiao et al., 2022, Shikhman et al., 2021).
• The M-index ($\text{MI}$), given by the sum of the quadratic index (number of negative Hessian eigenvalues on the critical cone) and the sparsity index ($k-\|x^*\|_0$), determines the cell structure in Morse theory and the number of saddles attached to level sets (Lämmel et al., 2022, Shikhman et al., 2021).
• Continuous reformulation regularizations, e.g., via additional constraints or objective smoothing, exactly preserve M-stationary points and their global topological properties, but the number of saddle points in the lifted space grows exponentially with problem dimension and sparsity slack, explaining inherent computational hardness (Lämmel et al., 2022).

The variational and Morse-theoretic approach clarifies the interplay between sparsity constraints and the critical point structure, guiding both theory and algorithms.

7. Relation to Other Stationarity Concepts and Applications

• Strong/KKT stationarity (CC-S) generally requires strong MPCC-type constraint qualifications, is too restrictive, and excludes many meaningful solutions in sparsity settings (Pang et al., 2021).
• Limiting (Mordukhovich) and Clarke stationarity coincide for $\ell_0$-constraints and capture all feasible directions, providing the correct first-order model for nonsmooth, union-of-subspaces sets (Xiao et al., 2022, Krulikovski et al., 2020).
• The theory applies broadly across compressed sensing, sparse learning, feature selection, signal processing, and combinatorial optimization where sparse solutions are sought under additional nonlinear constraints.
• The CC-M stationarity framework enables unification of continuous, sequential, and combinatorial perspectives, yielding practical, theoretically-guaranteed algorithmic frameworks and illuminating topological complexity in high-dimensional sparsity-constrained landscapes.

Cardinality-constrained Mordukhovich stationarity points are now central in both analysis and algorithm design for sparse optimization, providing verifiable, generic necessary conditions, bridging variational analysis and combinatorial sparsity, and enabling efficient algorithms that avoid the intractability of classical approaches.