Signature-Based Test Statistics

• Signature-based test statistics are methods that transform raw data into signature sequences to reveal hidden structures and detect deviations.
• They use techniques such as sorting, cumulative means, and weighted-runs to differentiate between unimodal and multimodal distributions with enhanced sensitivity.
• These methods offer computational efficiency and improved performance in clustering, anomaly detection, and local deviation analysis compared to traditional tests.

Signature-based test statistics form a class of statistical methods designed to detect structured deviations from a null hypothesis by transforming and summarizing raw data using specific signatures—mathematical constructions that compactly encode key features or ordered structure in the data. These statistics offer distinctive sensitivity to non-global departures (such as local anomalies or multimodality) and are increasingly utilized in clustering, anomaly detection, and hypothesis testing. Two prominent approaches include the Signature Test (Sigtest) for unimodality and the weighted-runs statistic for local deviation detection (Shahbaba et al., 2014, Beaujean et al., 2010).

1. Mathematical Formulation and Problem Setting

Signature-based tests operate by mapping a collection of observed scalar samples, often denoted as $y = [y_1,\dots,y_N]^T \in \mathbb{R}^N$, to a signature sequence designed to expose structure hidden in the original data. The typical hypothesis testing scenario involves a composite hypothesis, such as:

$$H_0: \text{The distribution of } y \text{ is unimodal} \quad \text{vs.} \quad H_1: \text{not unimodal}$$

or, in the context of evaluating local deviations,

$$H_0: X_i \sim \mathcal{N}(\mu_i, \sigma_i^2) \ \forall i,\quad H_1: \text{Existence of local structure or clusters of anomalous deviations}$$

Signature transformation techniques often exploit sorting, cumulative means, or runs of consecutive exceedances to reduce noise and concentrate statistical evidence relating to $H_1$. The transformation is governed by explicit forms such as order statistics, empirical quantiles, or cumulative averages.

2. Signature Test (Sigtest) for Unimodality

Sigtest specifically targets unimodality by transforming the data via absolute value sorting:

$$z = [z_1, \dots, z_N]^T = \text{sort}\left(|y|\right)$$

Two signatures are considered:

• Identity signature: $g_1(z_n) = z_n$
• Cumulative-mean signature: $g_2(z_n) = \frac{1}{n}\sum_{j=1}^n z_j$

These signatures are chosen because, for data drawn from a unimodal distribution, their sequence exhibits sharply reduced variance. The method exploits pointwise probabilistic envelopes:

$$U(n) = E[g_i(w_n)] + \gamma \sqrt{\operatorname{Var}(g_i(w_n))},\qquad L(n) = E[g_i(w_n)] - \gamma \sqrt{\operatorname{Var}(g_i(w_n))}$$

where $w_n$ are the ideal sorted absolute values under $H_0$, and $\gamma$ calibrates the width of the confidence band. The following decision rule is applied:

$$c_n = \begin{cases} 0 & \text{if } L(n) \leq g_i(z_n) \leq U(n) \ 1 & \text{otherwise} \end{cases},\qquad C = \frac{1}{N} \sum_{n=1}^N c_n$$

The null hypothesis $H_0$ is rejected (evidence for multi-modality) if $C > T$ for a chosen threshold $T$ (Shahbaba et al., 2014).

3. Weighted-Runs Test Statistic for Local Deviation Detection

Motivated by the insensitivity of classical global statistics (such as $\chi^2$) to clustering of deviations, the weighted-runs statistic tests for local, consecutive excesses:

• An observation $X_i > \mu_i$ is a "success", otherwise a "failure".
• The sequence is partitioned into alternating maximal runs of successes/failures.
• For each success run $A_j$, compute the run-weight:

$$w(A_j) = \sum_{X_i \in A_j} \frac{(X_i - \mu_i)^2}{\sigma_i^2}$$

• The test statistic is

$T = \max_{1 \leq j \leq M} w(A_j)$

where $M$ is the number of success runs. $T$ measures the size of the most extreme local deviation in the data, distinguishing it from the classical aggregate $\chi^2$ which is order-invariant (Beaujean et al., 2010).

4. Sampling Distributions, P-value Computation, and Calibration

Both approaches provide exact or approximate null distributions for hypothesis testing:

• Sigtest: Computes expected value and variance of the signature sequence under any unimodal distribution using classical order statistics theory. Confidence bands are calibrated deterministically via $\gamma$.
• Weighted-runs: The null distribution is derived via combinatorial summation over all integer partitions of the number of successes, accounting for all possible run-length profiles. Explicit formulas relate the cumulative probability:

$$P(T < T_{\text{obs}}\,|\,N) = \sum_{r=1}^N \sum_{M=1}^{\min(r,N-r+1)} \sum_{\pi \vdash r, |\pi|=M} \frac{W(\pi)}{2^N-1} \prod_{\ell=1}^r [F_{\chi^2_\ell}(T_{\text{obs}})]^{n_\ell}$$

where $n_\ell$ is the number of runs of length $\ell$ in the partition $\pi$ and $F_{\chi^2_\ell}$ is the chi-squared CDF. For $N \lesssim 80$, exact $p$-values are tractable; for larger $N$, Monte Carlo methods or empirical pre-tabulation are used (Beaujean et al., 2010).

5. Computational Complexity and Empirical Performance

• Sigtest: Sorting dominates with $O(N\log N)$ complexity. All signature and decision operations are $O(N)$. This is markedly faster than Hartigan’s dip ($O(N^2)$), KS, or AD tests, especially under calibration via resampling (Shahbaba et al., 2014).
• Weighted-runs: Enumeration of all integer partitions grows nearly exponentially, $\nu(N) = p(N+1)-1$ (where $p(n)$ is the partition function). For small to moderate $N$, this is feasible; for large $N$, the Monte Carlo approach offers $O(KN)$ complexity for $K$ simulations (Beaujean et al., 2010).

Empirical results demonstrate the higher sensitivity of signature-based statistics to local and multimodal features. For instance, in the detection of weakly separated normal mixtures, Sigtest's detection rate dominates that of AD, KS, and dip tests, with sub-millisecond average computation times. Weighted-runs achieves up to 30–40 percentage points higher power than global $\chi^2$ in detecting localized anomalies.

Test	2σ	2.25σ	2.5σ	2.8σ	3σ	Time (s)
Sigtest₂	56%	93%	99%	100%	100%	0.2×10⁻⁴
Sigtest₁	69%	97%	100%	100%	100%	0.2×10⁻⁴
AD	29%	76%	97%	100%	100%	3.96×10⁻⁴
KS	10%	37%	74%	95%	100%	30×10⁻⁴
Dip	3%	8%	21%	82%	94%	2197×10⁻⁴

6. Applications in Clustering and Anomaly Detection

Signature-based tests are integrated within top-down hierarchical clustering workflows as splitting criteria. Projected data from a candidate cluster undergoes a signature transformation and is tested for unimodality. If Sigtest or a weighted-runs statistic rejects the unimodal or i.i.d. null, the cluster is split (commonly via $k$-means). Empirical studies show that using Sigtest in place of classical methods yields improved clustering quality as measured by Variation of Information and Adjusted Rand Index. For example, on the Optical Digits UCI set, G-means$^+$ attains $VI=1.14\pm0.10$, $ARI=0.66\pm0.04$, compared to $VI=1.31\pm0.08$, $ARI=0.57\pm0.03$ for standard G-means (Shahbaba et al., 2014).

7. Theoretical Properties and Implementation Considerations

Theoretical justification derives from the variance-reducing effect of the signature mapping under the null hypothesis, which tightens confidence bands and magnifies sensitivity to departures (secondary modes, localized excesses). Practically, implementation requires careful numerical treatment (e.g., logarithmic evaluation of incomplete gamma functions for weighted-runs), and efficient algorithms for partition enumeration or simulation for larger data sizes. Both methods are robust to composite nulls if null simulations use fitted parameterizations, and can be implemented in a few dozens of lines in high-level scientific computing environments (Beaujean et al., 2010, Shahbaba et al., 2014).

A plausible implication is that signature-based statistics, given their computational efficiency and statistical power for structured alternatives, will continue to play a critical role in large-scale, high-dimensional inference. They are particularly well-suited as plug-in modules in clustering, signal segmentation, and local anomaly detection.