Constrained Agglomerative Hierarchical Clustering

• cAHC is a family of hierarchical clustering methods that incorporate spatial, temporal, or ontological constraints to restrict standard merge operations.
• The methodology adapts linkage criteria by introducing penalties and order-preserving schemes that integrate prior knowledge into the clustering process.
• Applications in genomics, spatial analysis, and product taxonomy demonstrate how constraints enhance interpretability and computational performance.

Constrained Agglomerative Hierarchical Clustering (cAHC) refers collectively to a family of hierarchical clustering methods in which the standard greedy agglomeration process is modified by the imposition of explicit constraints—structural, spatial, ontological, or order-theoretic—on the admissible cluster merges. These constraints alter both the solution space and the interpretability of the resulting dendrograms. cAHC is deployed in a variety of domains, including genomics, spatial analysis, product taxonomy construction, and scenarios with external ontologies or partial orders. Its theoretical properties, algorithmic foundations, and domain-specific adaptations have been rigorously characterized in recent research (Ma et al., 2018, Ambroise et al., 2019, Randriamihamison et al., 2019, Tzeng et al., 2022, Bakkelund, 2020).

1. Mathematical Framework and Taxonomy of Constraints

In classical agglomerative hierarchical clustering (AHC), each merge minimizes a linkage criterion over all unordered pairs of current clusters. cAHC restricts this by a constraint relation $R$ (contiguity, neighborhood, partial order, or prior ultrametric). Let $X = \{x_1,\ldots,x_n\}$ denote the data, $D=(d_{ij})$ the dissimilarity matrix, and $$\mathcal{G}^{(t)} = \{G_1^{(t)},\ldots,G_{k}^{(t)}\}$$ the clusters at stage $t$.

Types of Constraints

• Adjacency (contiguity) constraints: Clusters $G_u^{(t)}, G_v^{(t)}$ are eligible to merge only if $(G_u^{(t)},G_v^{(t)}) \in R$, typically representing spatial, temporal, or sequential adjacency (Ambroise et al., 2019, Tzeng et al., 2022, Randriamihamison et al., 2019).
• External ultrametric constraints: Merges are regularized towards agreement with a prior tree $T$, encoded via an ultrametric $u_T$, yielding a penalized dissimilarity $D'(i,j)=D_P(i,j)+\lambda u_T(i,j)$ (Ma et al., 2018).
• Order/partial order constraints: Merges are forbidden for pairs violating a prescribed order or DAG structure (e.g., "order-preserving" schemes), yielding partial dendrograms (Bakkelund, 2020).

The choice of $R$ or the constraint embedding fundamentally determines both the algorithmic mechanics and the theoretical guarantees.

2. Linkage Criteria Under Constraints

All cAHC variants reduce to modifying the set of eligible cluster pairs or integrating a penalty into the linkage function.

Canonical Formulations

• Standard Linkages: Single, complete, average, Ward's, and their Lance–Williams recursions are retained but restricted to pairs allowed by $R$ (Tzeng et al., 2022, Randriamihamison et al., 2019).
• Penalized/Regularized Linkage: For prior knowledge in the form of a tree $T$, the ultrametric $u_T$ is added as a convex penalty to the task-specific distance $D_P$, yielding $D'(i,j)$ as above (Ma et al., 2018).
• Ward’s linkage with arbitrary constraints:

$$\delta(G_u,G_v) = \frac{|G_u||G_v|}{|G_u|+|G_v|} \left(\frac{\tilde \Delta(G_u,G_v)}{|G_u||G_v|} - \frac{\tilde \Delta(G_u,G_u)}{2|G_u|^2} - \frac{\tilde \Delta(G_v,G_v)}{2|G_v|^2}\right)$$

but only $(G_u,G_v)\in R$ are ever merged (Randriamihamison et al., 2019).

Specialized algorithms exploit properties of certain constraint types, e.g., spatial adjacency or band-matrix similarity for computational gains (Ambroise et al., 2019).

3. Algorithmic Procedures and Complexity

The imposition of constraints alters both the computational cost and workflow of agglomerative clustering.

General Algorithmic Structure

1. Initialization: Each datum forms a singleton cluster; constraint structure (adjacency graph, ultrametric penalties, DAG) is constructed (Ambroise et al., 2019, Tzeng et al., 2022, Bakkelund, 2020).
2. Iterative Merging: At each step, among all admissible pairs $(G_u,G_v)\in R$, select the pair minimizing the linkage criterion (possibly penalized for prior knowledge as in $D'$).
3. Constraint Update: After merging $G_u$ and $G_v$ into $G_{uv}$, update $R_t$ or adjacency matrices per the rules of the constraint type. In prior-based cAHC, $R$ is implicit but $D'$ is recomputed for all pairs (Ma et al., 2018).
4. Termination: Stop when only one cluster remains or when no further admissible merges are possible (for order-preserving cases, yielding partial dendrograms).

Complexity

• General cAHC with contiguity constraints: For sparse $R$ (e.g., planar or linear adjacency), candidate merges per iteration reduce from $O(n^2)$ to $O(n)$, yielding total $O(n^2)$ complexity or better (Ambroise et al., 2019, Tzeng et al., 2022).
• Ultrametric penalty schemes: Cost dominated by $O(n^2 h + n^2 \log n)$, where $h$ is the prior tree height; efficient for moderate $n$, with preclustering possible for large $n$ (Ma et al., 2018).
• Order-preserving DAG-based cAHC: Worst-case $O(n^5)$ for general partial orders, but can be $O(K n^3)$ when the order is sparse (Bakkelund, 2020).

Enhanced data structures—priority queues for adjacency constraints, precomputed pencil sums for banded similarity—yield substantial practical accelerations (Ambroise et al., 2019).

4. Theoretical Properties and Guarantees

Monotonicity and Ultrametricity

• Unconstrained AHC/Ward: Merge heights are non-decreasing; the induced cophenetic distance is an ultrametric (Randriamihamison et al., 2019).
• cAHC: General constraints may break monotonicity, producing "crossovers" (merge at lower height than a child). For spatial or linear adjacencies, monotonicity is likely to hold if the constraint is compatible with the data (Randriamihamison et al., 2019).
• Ultrametric Penalty (Prior constraints): For sufficiently large penalty $\lambda$, the method exactly recovers the prior tree, and single-linkage ensures stability and permutation invariance by Gromov–Hausdorff continuity (Ma et al., 2018).
• Order-preserving cAHC: Algorithmic merging of non-comparable blocks guarantees order preservation; the induced (partial) dendrograms can be mapped exactly into ultrametric space (Bakkelund, 2020).

Correctness and Approximation

• Adjacency constraints: cAHC merges precisely the same pairs as naïve adjacency-constrained schemes; Lance–Williams formulae assure exact linkage values (Ambroise et al., 2019).
• NP-hardness: For complete linkage under order constraints, global optimum is NP-hard, although sampling/randomized tie resolution in moderate $n$ suffices empirically (Bakkelund, 2020).

5. Applications and Empirical Evaluations

Taxonomy Construction with Prior Knowledge (Amazon Case Study)

cAHC is applied to construct a customer behavior-based product taxonomy, penalizing deviations from an existing ontological browse hierarchy (the prior tree $T$) (Ma et al., 2018). Task-specific dissimilarity is computed using LDA-derived vector embeddings from customer logs, combined with the ultrametric of $T$. Adjusting $\lambda$ enables interpolation between a purely data-driven dendrogram and strict adherence to the prior tree. Empirical results show that intermediate $\lambda$ values maximize cluster purity and minimize entropy, outperforming both no-constraint and hard-prior baselines.

Genomics (GWAS and Hi-C)

Adjacency-constrained cAHC partitions chromosomes into ordered, contiguous LD blocks or topologically associating domains. The band-similarity assumption ($s_{ij}=0$ for $|i-j|\geq h$) permits near-linear time algorithms. In both GWAS and Hi-C, cAHC supports high-resolution, interpretable segmentations that correspond to biological structure, with domain-informed model selection criteria guiding the optimal number of clusters (Ambroise et al., 2019).

Spatial Data Analysis

Spatial contiguity-constrained cAHC, as implemented in HCV, is suited for segmenting areal or tessellated point data into geographically contiguous and feature-homogeneous regions. Customized indices (Spatial Mixture Index, M3C consensus) replace classical criteria for choosing the number of clusters, ensuring spatial coherence (Tzeng et al., 2022).

Order-Preserving Clustering

cAHC for strict partial orders or DAGs (e.g., part-of hierarchies) produces clusterings strictly respecting the initial ordering, yielding forests of dendrograms or partial dendrograms. This approach is essential in databases where merging across order-induced boundaries is disallowed (e.g., project task dependencies, part assembly orders) (Bakkelund, 2020).

6. Practical Considerations and Limitations

• Interpretability: cAHC enhances interpretability when constraints match domain knowledge (geospatial, ontological, sequential) but may yield artifacts or degenerate structures if imposed inappropriately (numerous crossovers, reversals) (Randriamihamison et al., 2019).
• Monotonicity Violation: Researchers should visualize merge heights and crossovers. For critical applications, alternative height functions (e.g., within-cluster inertia) can be plotted to ensure monotone dendrograms (Randriamihamison et al., 2019).
• Choice of Parameters: Penalty parameters (e.g., $\lambda$ in ultrametric-penalized cAHC) require cross-validation; model selection for $K$ clusters may employ domain-tuned indices rather than generic silhouette/gap methods (Ma et al., 2018, Tzeng et al., 2022).
• Computational Efficiency: For sparse or structured constraints, cAHC can yield substantial computational savings compared to unconstrained AHC; in spatial/sparse domains, complexity gains are significant (Ambroise et al., 2019, Tzeng et al., 2022).
• Algorithm Selection: Where the constraint matches expected structure (adjacency, order), cAHC is preferred. Unconstrained AHC may outperform cAHC when strong non-local clusters exist or when the domain constraint is misaligned (Randriamihamison et al., 2019).

7. Summary Table: cAHC Constraint Types

Constraint Type	Typical Domain	Algorithmic Modification
Adjacency/Contiguity	Genomics, spatial, time	Only contiguous clusters merged
Ultrametric (prior)	Taxonomy, ontology	Penalized distance: $D'+\lambda u_T$
Spatial adjacency	Areal spatial data	Cluster adjacency in graph $A$
Order/DAG	Task, process orders	Only non-comparable pairs merged

Each cAHC variant leverages domain-specific structure to restrict the space of agglomerations, trading off global optimality for interpretability, computational gains, and adherence to auxiliary knowledge or constraints. The theoretical and empirical studies across applications (Amazon taxonomies, GWAS, Hi-C, spatial regions, industrial part orderings) confirm the flexibility and domain value of the cAHC paradigm (Ma et al., 2018, Ambroise et al., 2019, Tzeng et al., 2022, Randriamihamison et al., 2019, Bakkelund, 2020).