Unnormalized Spectral Clustering

• Unnormalized spectral clustering is a graph-based algorithm that segments data by analyzing the eigenstructure of the Laplacian matrix derived from the similarity graph.
• The method constructs a similarity graph, computes eigenvectors of L = D - W, and applies k-means on the spectral embedding to recover clusters.
• Despite its clear linear-algebraic foundations and theoretical guarantees, the approach can be sensitive to degree heterogeneity compared to normalized methods.

Unnormalized spectral clustering is a graph-based algorithmic framework for partitioning data into clusters by leveraging the eigenstructure of the unnormalized graph Laplacian matrix. It directly relaxes combinatorial graph-cut objectives and embeds data points into a low-dimensional spectral space where geometric separation reflects underlying cluster structure. The method emphasizes the topology of the constructed similarity graph, using linear algebraic relaxations for computational feasibility, and has well-documented theoretical and practical characteristics in both general and model-based data regimes.

1. Definition of the Unnormalized Laplacian and Graph Construction

Given data points $x_1, \dots, x_n \in \mathbb{R}^d$ and a nonnegative, symmetric similarity function $s(x_i, x_j)$, an undirected similarity graph $G=(V,E)$ is constructed where $V = \{v_1, \dots, v_n\}$ and edge weights are $w_{ij}=s(x_i, x_j) \geq 0$ with $w_{ij}=w_{ji}$ and $w_{ii}=0$ (0711.0189). Several sparsification schemes are common, such as $k$-nearest neighbor, $\varepsilon$-radius, or fully-connected Gaussian-weighted graphs. The adjacency matrix $W=(w_{ij})$, and diagonal degree matrix $D$ with $D_{ii}=\sum_j w_{ij}$, are defined. The unnormalized graph Laplacian is then

$L = D - W.$

This matrix is symmetric, positive semidefinite, and satisfies $Lf = 0$ for constant vectors $f$ (0711.0189). The fundamental quadratic form is

$$f^\top L f = \frac{1}{2} \sum_{i,j=1}^n w_{ij} (f_i - f_j)^2,$$

which encodes the connectivity structure of $G$.

2. Algorithmic Workflow and Spectral Relaxation

The unnormalized spectral clustering algorithm proceeds as follows (0711.0189):

1. Graph Construction: Compute $W$ by the selected similarity and sparsification.
2. Degree and Laplacian: Form $D$ and $L = D-W$.
3. Spectral Decomposition: Solve $L u_\ell = \lambda_\ell u_\ell$ for eigenpairs $(\lambda_\ell, u_\ell)$ and extract the $k$ eigenvectors with the smallest eigenvalues.
4. Spectral Embedding: Treat each data-point $i$ as the $i$-th row of the $n \times k$ eigenvector matrix $U$.
5. Clustering Assignment: Run $k$-means in $\mathbb{R}^k$ on the embeddings.
6. Cluster Recovery: Assign points to clusters according to $k$-means output.

The algorithm is a relaxation of the RatioCut objective, where minimizing the sum over clusters $\sum_j \frac{\mathrm{Cut}(A_j, \bar{A}_j)}{|A_j|}$ is NP-hard. By relaxing indicator vectors to real vectors with orthogonality and norm constraints, the problem reduces to computing the bottom $k$ eigenvectors of $L$ (0711.0189).

3. Theoretical Guarantees and Consistency

The spectral properties of $L$ directly encode cluster structure: the multiplicity of the zero eigenvalue equals the number of connected components in the graph, with corresponding eigenvectors as component indicators (0711.0189). For i.i.d. samples $x_1,\dots,x_n$ from an underlying measure $\rho$ on $D\subset\mathbb{R}^d$, and with similarity graph constructed using an appropriate kernel $\eta$ and connectivity radius $\varepsilon_n \to 0$, the following holds (Trillos et al., 2015):

• Eigenvalue Convergence: For each fixed $k$,

$$\frac{2\lambda_k^{(n)}}{n \varepsilon_n^2} \to \sigma_\eta \lambda_k$$

where $\lambda_k$ is the $k$th eigenvalue of a continuum differential operator, and $\sigma_\eta$ depends on the kernel.

• Eigenvector Convergence: For unit $D$-norm eigenvectors $u_k^{(n)}$ of $L$, the functions $u_k^{(n)}$ converge, in the $TL^2$ topology, to continuum eigenfunctions $u_k$.
• Cluster Consistency: If the $k$-means algorithm is run on the embedding by the first $k$ eigenvectors, the resulting clusters converge (weakly, in measure) to the continuum partition induced by $(u_1,\dots,u_k)_{\#}\rho$, under assumptions on graph connectivity and scaling of $\varepsilon_n$.

A $\Gamma$-convergence analysis establishes that discrete graph Dirichlet energies converge to the continuum Dirichlet energy, directly linking spectral clustering on finite data with the underlying population structure. Explicit scaling conditions on $\varepsilon_n$ ensure the spectral limits are meaningful: $\varepsilon_n \approx (\log n / n)^{1/d}$ is sufficient, and the method remains consistent up to the connectivity threshold (Trillos et al., 2015).

4. Model-Selection, Parameterization, and Practical Implementation

Unnormalized spectral clustering requires careful graph construction and parameter tuning. For geometric data, a topological approach (Rieser, 2015) constructs a one-parameter family of graphs $\{G_r\}_{r \geq 0}$ by thresholding ambient distances at scale $r$. The correct $r$ is selected using two data-driven criteria:

• Average Relative Neighborhood Volume: $$R_{G_r} = \frac{1}{|Z|} \sum_v \frac{|N(v)|}{|C(v)|}$$, minimized over $r$.
• Average Relative Entropy: $H_{r,1}$ averages over nodes the Kullback–Leibler divergence between heat-diffused distributions at time $t=1$ and steady-state within components, maximized over $r$.

Cluster assignment is then obtained by extracting the kernel of $L_{\hat r}$ and assigning points by projection in the space of 0-eigenvectors (Rieser, 2015). Computationally, for $n$ data points and $m$ candidate $r$ values, complexity is $O(mn^3)$ in the worst case, though iterative eigensolvers and sparse-matrix methods reduce practical costs.

5. Comparison to Normalized Spectral Clustering

Unnormalized and normalized spectral clustering share the foundational use of graph-Laplacian eigenvectors but differ in normalization and objective. The unnormalized Laplacian $L=D-W$ relaxes the RatioCut, which depends on the cardinality of clusters $|A_j|$, while normalized methods ($L_{\text{sym}} = D^{-1/2} L D^{-1/2}$, $L_{\text{rw}} = I - D^{-1} W$) target the Normalized Cut objective, accounting for cluster volumes $\operatorname{vol}(A_j)$. Several key differences are documented (0711.0189, Sarkar et al., 2013):

• Consistency: Unnormalized spectral clustering may lack statistical consistency for large graphs unless all $d_i$ are bounded away from zero and the spectrum considered remains well below $\min_i d_i$. Eigenvectors corresponding to higher eigenvalues can become localized and uninformative.
• Degree Sensitivity: Unnormalized algorithms are sensitive to degree heterogeneity. If degrees vary widely or some vertices have small degree, eigenvectors can behave pathologically.
• Empirical Performance: Both normalized and unnormalized methods achieve the same asymptotic rate of convergence for misclassification in stochastic blockmodels, but normalization consistently shrinks within-cluster spread in the spectral embedding by a constant factor, leading to lower error in finite samples and on real-data link-prediction tasks. For example, normalized clustering attained lower misclassification rates than unnormalized in co-authorship and political blog datasets; for instance, normalized SC misclassified 4% of blogs versus 37% for unnormalized SC after preprocessing (Sarkar et al., 2013).
• When Unnormalized Clustering Fails: Pathological cases exist (e.g., cockroach graphs) where unnormalized spectral clustering produces suboptimal partitions while normalized methods succeed.

6. Applications, Advantages, and Pitfalls

Unnormalized spectral clustering is widely applicable due to its algorithmic simplicity and its close connection to linear algebra and graph theory. It is favored when the cluster size is of interest, the graph is well-behaved (relatively uniform degree), and the RatioCut objective is appropriate. Its advantages are: direct computation on $L$, algorithmic clarity, and connection with indicator subspaces for connected components (0711.0189).

However, sensitive dependence on the global graph structure and lack of cluster-volume normalization can lead to poor performance on graphs with unbalanced or heterogeneous degree distributions, as well as statistical inconsistency in large sample limits under mild connectivity violations. Empirically, normalized variants often outperform unnormalized, especially in the presence of degree heterogeneity and for moderate $n$ (Sarkar et al., 2013).

A plausible implication is that, for rigorous statistical consistency and robust finite-sample behavior, normalized spectral clustering should be preferred under moderate or unknown degree variation. Unnormalized clustering remains useful for pedagogical purposes, for balanced geometric data, and as the foundation of topological or parameter-free spectral methods (Rieser, 2015).

7. Summary Table: Key Properties

Property	Unnormalized Spectral Clustering	Normalized Spectral Clustering
Laplacian definition	$L = D-W$	$L_{\rm sym} = D^{-1/2}LD^{-1/2}$
Balances by	Cluster size ($|A_j|$)	Cluster volume ($\operatorname{vol}(A_j)$)
Statistical consistency	Only in restricted settings (high min degree, small spectrum)	Robust as $n\to\infty$
Sensitivity to degree heterogeneity	High	Low
Empirical neighbor spread	Larger within-cluster spread	Shrunk by constant factor
Typical use cases	Uniform geometric data, topological settings	Heterogeneous graphs, real data analysis

The selection of unnormalized spectral clustering should be informed by graph structure, application requirements, and theoretical guarantees. In asymptotic and real-data regimes with degree variability or in the presence of sparse clusters, normalized methodologies are often statistically and empirically superior. Nonetheless, unnormalized variants offer insight into the interplay between topology, spectral theory, and combinatorial clustering (Rieser, 2015, Trillos et al., 2015, 0711.0189, Sarkar et al., 2013).