Near-Convex Archetypal Analysis

Abstract: Nonnegative matrix factorization (NMF) is a widely used linear dimensionality reduction technique for nonnegative data. NMF requires that each data point is approximated by a convex combination of basis elements. Archetypal analysis (AA), also referred to as convex NMF, is a well-known NMF variant imposing that the basis elements are themselves convex combinations of the data points. AA has the advantage to be more interpretable than NMF because the basis elements are directly constructed from the data points. However, it usually suffers from a high data fitting error because the basis elements are constrained to be contained in the convex cone of the data points. In this letter, we introduce near-convex archetypal analysis (NCAA) which combines the advantages of both AA and NMF. As for AA, the basis vectors are required to be linear combinations of the data points and hence are easily interpretable. As for NMF, the additional flexibility in choosing the basis elements allows NCAA to have a low data fitting error. We show that NCAA compares favorably with a state-of-the-art minimum-volume NMF method on synthetic datasets and on a real-world hyperspectral image.

• The paper introduces NCAA, a flexible model that relaxes the strict convexity constraint via an adjustable margin, achieving improved interpretable archetype recovery with minimal error.
• It develops a two-block coordinate descent algorithm with Nesterov acceleration to efficiently optimize matrices under generalized simplex constraints.
• Empirical results on synthetic and hyperspectral datasets demonstrate NCAA’s superior performance in mean spectral angle error compared to methods like MinVolNMF and SNPA.

Near-Convex Archetypal Analysis: Synthesis, Algorithms, and Empirical Assessment

Motivation and Context

Traditional Nonnegative Matrix Factorization (NMF) offers a computationally tractable and interpretable framework for dimensionality reduction in nonnegative data matrices, with wide deployment in areas such as hyperspectral unmixing. However, NMF's interpretability is limited because basis elements are not guaranteed to reflect actual data points; archetypal analysis (AA) and convex NMF address this by restricting basis vectors to reside in the convex hull of data points. Despite enhanced interpretability, this constraint often results in increased approximation error, especially in highly mixed or noisy settings. Prior relaxations—such as introducing convexity margins or trade-offs between reconstruction error and archetype proximity—have not fully reconciled the interpretability-versus-error trade-off, nor have they supported scalable inference with explicit coefficient matrices linking archetypes to data.

The Near-Convex Archetypal Analysis (NCAA) model proposed in this paper introduces a relaxation of the convexity constraint via an adjustable margin $\epsilon$, yielding improved estimation accuracy while preserving the interpretability of AA.

Model Formulation

The NCAA objective constrains archetypes to near-convex combinations (NCCs) of selected data points, allowing basis vectors to lie outside the strict convex hull via negative weights bounded by $-\epsilon$. The formulation is as follows: for a data matrix $X \in \mathbb{R}^{m \times n}$, subset matrix $Y \in \mathbb{R}^{m \times d}$ (collected via SNPA or hierarchical clustering), factorization rank $r$, and scalar $\epsilon$, NCAA seeks matrices $A \in \mathbb{R}^{d \times r}$ and $H \in \mathbb{R}^{r \times n}$ to minimize $\Vert X - Y A H \Vert_F^2$, subject to $A(k,l) \geq -\epsilon$ and $\sum_{k=1}^d A(k, l) = 1$, with $H$ columns in the unit simplex.

The geometric implications are formalized: as $\epsilon$ increases, the archetypes $W = YA$ are pushed further from the convex hull of $Y$, parameterizing the trade-off between interpretability and fidelity. The purity level $p$ quantifies the separation between archetypes and data points, guiding $\epsilon$ selection and indicating when exact recovery (separable NMF) is possible.

Optimization Strategy

NCAA is solved via a two-block coordinate descent (BCD), alternating projected gradient steps for $A$ and $H$. Projections onto the generalized simplex (for $A$ with negative margins, and $H$ onto the standard simplex) employ fast projected gradient methods with Nesterov acceleration and adaptive line search for step size.

Algorithmic tuning of $\epsilon$ is achieved by initializing at a small value and doubling until the reduction in reconstruction error stalls, ensuring data-adaptive flexibility. The authors also propose column-wise fine-tuning of $\epsilon_l$ for each archetype, moving archetypes toward the convex hull as long as the increase in approximation error remains marginal.

The computational complexity of NCAA is linear in input dimensions per iteration ($\mathcal{O}(mnr)$), provided $d$ is chosen as a small multiple of $r$, making the method viable for large-scale applications.

Empirical Results

Synthetic Data Evaluation

NCAA is benchmarked against state-of-the-art minimum-volume NMF (MinVolNMF with logdet penalty and simplex constraints) and SNPA on synthetic datasets parametrized by purity $p$, rank $r$, and noise $\upsilon$. The performance metric is Mean Removed Spectral Angle (MRSA), with lower values signifying better recovery. NCAA attains substantially lower MRSA than MinVolNMF and SNPA across most configurations, including low purity, high rank, and high noise scenarios. Specifically, SNPA is only competitive in ideal separable settings ($p=1$), confirming NCAA's robustness under practical, non-ideal distributions.

Hyperspectral Unmixing

On a real hyperspectral image (HYDICE Urban), NCAA (with hierarchical clustering for $Y$ and fine-tuned $\epsilon$) recovers spectral signatures closely matching ground truth endmembers. The abundance maps produced align with physical expectations, and NCAA outperforms MinVolNMF in MRSA (5.56 vs 5.73) and qualitative recovery of material mixtures. This demonstrates NCAA's practical applicability and interpretability in real-world geometric unmixing.

Implications and Future Directions

NCAA's relaxation of AA's convexity constraint formalizes the balance between interpretability and estimation error, enabling practitioners to tune fidelity via $\epsilon$ and supporting scalable inference with explicit archetype-to-data mappings. The adaptive tuning of $\epsilon$—both globally and per-archetype—addresses variability in data purity and ensures robustness to non-separability.

The theoretical implications extend to identifiability: the proximity to minimum-volume NMF suggests that NCAA can inherit theoretical guarantees, such as uniqueness under noisy or mixed scenarios. Future work includes analysis of NCAA's uniqueness properties, extension to models with learnable $Y$, incorporation of row-sparsity constraints for automatic selection of archetypes, and alternate regularization schemes in the objective function.

Conclusion

The NCAA framework advances NMF-based dimensionality reduction by unifying interpretable archetype construction with low reconstruction error. Its geometric and algorithmic innovations grant practitioners both robustness and clarity in factorization, as confirmed by strong empirical results in synthetic and real data scenarios. NCAA lays the groundwork for further developments in interpretable matrix factorization, adaptive convexity relaxation, and identifiability theory.