Dependence-Aware KCPD Theory

• The paper introduces a dependence-aware framework for KCPD that formalizes m-dependence to capture local correlations in sequential data.
• It employs rigorous concentration analysis via Janson's inequality to derive oracle inequalities and robust segmentation guarantees.
• The theory bridges statistical segmentation with process calculi, enabling precise change-point localization and extending applicability to language data.

Dependence-aware theory for Kernel Change-Point Detection (KCPD) addresses the key challenge of statistical inference and segmentation under dependence structures intrinsic to real-world sequential data, such as text, where observations cannot be assumed independent. By formalizing and analyzing KCPD under $m$-dependent sequences—a finite-memory model capturing short-range dependence—the theory enables nonparametric consistency results and robust segmentation guarantees applicable to language and other domains exhibiting local correlation. The dependence-aware framework further develops connections to reversible process calculi, embedding structural relations like dependence, independence, and causality directly into the detection paradigm.

1. The $m$-Dependence Model

The $m$-dependence framework posits that a sequence $(Y_t)_{t=1}^T$ is $m$-dependent if any two non-overlapping blocks separated by more than $m$ indices are probabilistically independent. Specifically, for $|t'-t|>m$, $Y_t$ and $Y_{t'}$ are independent. This model is well-suited for text, where contextual dependencies decay beyond a short window. It retains sufficient complexity to model linguistic phenomena—such as local discourse coherence—while remaining analytically tractable for concentration and consistency analysis in the KCPD setting (Jia et al., 26 Jan 2026, Diaz-Rodriguez et al., 3 Oct 2025).

Formal Definition

Let $(Y_t)_{t=1}^T$ denote a sequence of random variables. The sequence is $m$-dependent if, for all $t < t'$ such that $|t'-t| > m$, the $\sigma$-algebras generated by $\{Y_s : s \leq t\}$ and $\{Y_s : s \geq t'\}$ are independent. This finite-memory assumption captures the prevalence of strong short-range, but negligible long-range, correlations in natural language data.

2. KCPD Objective and Penalized Population Risk

Given a sequence of embeddings $Y_1, \dots, Y_T \in \mathbb{R}^d$ and a bounded, characteristic kernel $$k: \mathbb{R}^d \times \mathbb{R}^d \rightarrow [0, M]$$ with associated RKHS $\mathcal H$ and feature map $\phi$, the population segment cost for $[s, e]$ is

$$C(s,e) := \mathbb{E}\left[ \widehat{C}(s,e) \right] = \sum_{t=s}^e \mathbb{E}[k(Y_t,Y_t)] - \frac{1}{e-s+1}\sum_{i,j=s}^e \mathbb{E}[k(Y_i,Y_j)],$$

where $\widehat{C}(s,e)$ is the empirical within-segment RKHS scatter. For a candidate segmentation $\boldsymbol\tau'_{K'}$ of length $K'$, the penalized population risk is

$$L^\star(\boldsymbol\tau'_{K'}) := \sum_{k=1}^{K'+1} C(\tau'_{k-1}+1, \tau'_k) + \beta_T K',$$

with a penalty parameter $\beta_T$ to control over-segmentation. For $m$-dependence, it is required that

$$\beta_T \geq 16 M \sqrt{2(8m+5) T \log T} + 2M(1+6m), \qquad \beta_T = O(\sqrt{T \log T}).$$

3. Statistical Guarantees: Oracle Inequality and Localization

Oracle Inequality

Let $(Y_t)$ be $m$-dependent and piecewise stationary with bounded characteristic kernel $k$. The empirical KCPD estimator

$$\widehat{\boldsymbol\tau}_{\widehat K} = \arg\min_{\boldsymbol\tau'} \left\{ \sum_{k=1}^{K'+1} \widehat C(\tau'_{k-1}+1,\tau'_k) + \beta_T K' \right\}$$

satisfies, with probability at least $1-T^{-1}$,

$$\sum_{k=1}^{\widehat K + 1} C(\widehat\tau_{k-1} + 1, \widehat\tau_k) + \beta_T \widehat K \le \inf_{\boldsymbol\tau'_{K'}} \left[ \sum_{k=1}^{K'+1} C(\tau'_{k-1}+1, \tau'_k) + \beta_T K' \right] + 2 \lambda_T T,$$

with $\lambda_T = 4\sqrt{2} M \sqrt{(8m+5)\log T}$. This inequality bounds the estimator's (population) penalized risk by the optimal attainable risk, up to a $O(T\sqrt{\log T})$ excess term that is only mildly inflated by $m$-dependence (Jia et al., 26 Jan 2026, Diaz-Rodriguez et al., 3 Oct 2025).

Localization Guarantee

Under further assumptions: detectability ($$\Delta_\star^2 := \min_k \| \mu_{P_k} - \mu_{P_{k+1}} \|_\mathcal{H}^2 > 0$$), minimum spacing ($\ell_T / \sqrt{T \log T} \to \infty$), and signal dominance on mixed intervals, every true change point $\tau_k^\star$ is recovered by the estimator within a window of size $\delta_T = O(\sqrt{T \log T})$, which is negligible compared to $\ell_T$ as $T \to \infty$. Explicitly,

$$\Pr \left( \forall 1 \le k \le K: \min_{0 \le j \le \widehat K} |\widehat\tau_j - \tau_k^\star| \le \delta_T \right) \to 1.$$

Thus, KCPD under $m$-dependence achieves nonparametric consistency both in the number and (in a weak sense) the location of change points as $T$ increases.

4. Proof Techniques and Theoretical Machinery

The dependence-aware theory leverages several foundational tools:

• Uniform deviation of empirical RKHS costs $\widehat{C}(s,e)$ from their expectation $C(s,e)$ is obtained by applying Janson's inequality on dependency graphs with chromatic number $O(mn)$. This yields exponential concentration and supports a union bound over all $O(T^2)$ segments.
• The non-oversegmentation result relies on stability: no subdivision of a homogeneous segment can decrease penalized risk, due to concentration and the lower bound on $\beta_T$.
• In mixed intervals, careful lower bounding of segment cost reductions justifies that failing to estimate a true change incurs a detectable excess risk, thus enforcing location consistency.
• $m$-dependence is essential in both the concentration analysis (controlling the effective variance via dependency graph methods) and in the population cost expansion (factorizing off-diagonal kernel terms beyond lag $m$).

A plausible implication is that these concentration tools could be extended to more general dependence structures, such as $\alpha$-mixing or $\rho$-mixing, although this remains an open direction (Jia et al., 26 Jan 2026, Diaz-Rodriguez et al., 3 Oct 2025).

5. Simulation and Empirical Validation

To empirically validate dependence-aware KCPD, synthetic documents were generated by prompting LLMs (GPT-4.1) to write sequentially in an $m$-order Markov manner (conditioning each sentence on the previous $m$). These synthetic sequences, with known boundaries and controlled $m$, serve as testbeds to:

• Verify that segmentation errors (as measured by $P_k$ error and WindowDiff) decrease as document length $T$ increases, consistent with the theory's $O(\sqrt{T \log T})$ window scaling.
• Confirm that the prescribed penalty scaling for $\beta_T$ ensures robust performance.
• Demonstrate practical segmentation reliability on both synthetic and real data, including Choi's synthetic benchmark, Wikipedia, arXiv abstracts, and Taylor Swift's tweets (Jia et al., 26 Jan 2026, Diaz-Rodriguez et al., 3 Oct 2025).

Table: Simulation Design Elements

Aspect	Specification	Purpose
Text Generation	GPT-4.1, $m$-Markov conditioning ($m\in\{10,20,30\}$)	Enforce $m$-dependence
Segmentation	$K \approx 2\log T$ true change points	Mirror theoretical model
Evaluation	$P_k$ error, WindowDiff metrics	Quantify segmentation accuracy
Embeddings	sBERT, MPNet, OpenAI text-embedding-3	Test across modern text embedding models

6. Structural Dependence: Process Calculi and Bisimulation

While the statistical theory of KCPD addresses dependence via $m$-dependent random sequences, dependence-aware semantics has also been formalized for process calculi—systems modeling concurrent computations using labeled transition systems with communication keys and proof labels (Aubert et al., 2024). In this setting:

• Dependence and independence relations between proof labels or transitions are formalized and shown to be complementary on connected transitions (Theorem 8).
• Canonicity results guarantee uniqueness of independence relations and thus of derived causality and conflict.
• Key-preserving (KP) and dependence-preserving (DP) bisimulations offer behavioral equivalence notions; for standard processes, KP and DP bisimulations coincide (Theorem 28).

A plausible implication is that such semantic notions can be instantiated analogously in KCPD frameworks, with keys representing segment boundaries and dependency relations controlling the granularity and compositionality of change-point detection.

7. Limitations and Open Problems

Current dependence-aware KCPD is limited by the strictness of the $m$-dependence assumption—real text may exhibit decaying, not finite, memory. Extending theoretical guarantees to more realistic dependence structures such as $\alpha$-mixing or $\rho$-mixing sequences remains an open direction. Additionally:

• The penalty parameter $\beta_T$ and window size $\delta_T$ are conservatively set via worst-case uniform concentration; tighter or adaptive selection under dependence is not yet established.
• Theoretical analysis presumes characteristic kernel functions, whereas in practice non-characteristic kernels (e.g., cosine similarity) may outperform or be preferred in NLP applications—establishing dependence-aware theory for such kernels is unresolved.
• Long-range dependence (such as topic drift) may necessitate new statistical tools, such as self-normalization or block bootstrap (Jia et al., 26 Jan 2026).

The dependence-aware theory for KCPD provides the first comprehensive nonparametric consistency analysis and empirical foundation for segmentation under short-range dependence, unifying concentration, risk bounds, and localization guarantees (Jia et al., 26 Jan 2026, Diaz-Rodriguez et al., 3 Oct 2025). The structural approaches from the process calculi literature further invite extensions to compositional and semantic analyses of dependency in KCPD systems (Aubert et al., 2024).