HDP-HSMM: Bayesian Model with Explicit Durations

• HDP-HSMM is a Bayesian nonparametric time-series model that generalizes HMMs by incorporating explicit, arbitrary state-duration distributions.
• It addresses the geometric sojourn-time limitation of conventional models, leading to improved segmentation in applications like speaker diarization and driving pattern analysis.
• Efficient inference is achieved via blocked Gibbs samplers and robust variants mitigate over-segmentation by merging redundant states.

A Hierarchical Dirichlet Process Hidden Semi-Markov Model (HDP-HSMM) is a Bayesian nonparametric time-series model that generalizes both the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) and traditional hidden semi-Markov models (HSMMs). It allows an unbounded number of latent states and, crucially, supports arbitrary explicit state-duration distributions, overcoming the geometric sojourn-time limitation of conventional HMMs and HDP-HMMs. The model is particularly suited for sequential data where the temporal dynamics exhibit non-geometric segment lengths and where the number of dynamical regimes is unknown or potentially infinite (Johnson et al., 2012, Johnson et al., 2012, Wang et al., 2017, Aguirre et al., 2023, Taniguchi et al., 2015).

1. Generative Construction and Hierarchical DP Priors

The HDP-HSMM extends the HDP-HMM by enriching the Markov process over hidden states with explicit, state-dependent duration distributions. The generative process is as follows:

• Global stick-breaking (GEM) for base state weights:

$$v_k \sim \mathrm{Beta}(1, \gamma), \quad \beta_k = v_k \prod_{\ell<k} (1 - v_\ell)$$

$\beta = (\beta_k)_{k=1}^\infty$ forms a random probability measure over the (potentially infinite) set of states.

• State-specific transition measures:

$\pi_j \sim \mathrm{DP}(\alpha, \beta)$

Each $\pi_j$ is a probability vector governing transitions out of state $j$, with base measure $\beta$.

• State-specific emission and duration parameters:

$$\theta_j \sim H_\theta, \quad \omega_j \sim H_D, \quad D_s \mid z_s = j, \omega_j \sim H_D(\omega_j)$$

$\theta_j$ parametrizes the emission distribution, $H_\theta$; $\omega_j$ parametrizes the duration law $H_D$ (e.g., Poisson, negative-binomial, or any explicit non-geometric dwell law).

• Sequential Generation:
  • Select the next state $z_s \sim \pi_{z_{s-1}}$, with $z_0$ as a dummy start state.
  • Draw a segment length $D_s \sim H_D(\omega_{z_s})$.
  • Emit $D_s$ observations i.i.d. from $F(\theta_{z_s})$: $y_{t_{s-1}+1:t_s} \sim F(\theta_{z_s})$, where $t_s = \sum_{r=1}^s D_r$.
• Joint Probability:

$$\begin{aligned} &p(\{\beta_k\},\{\pi_j\},\{\omega_j\},\{\theta_j\},\{z_s, D_s\}_{s=1}^S, y_{1:T}) \ &= \left[\prod_{k=1}^\infty p(\beta_k)\right] \prod_{j=1}^\infty p(\pi_j | \beta) p(\omega_j) p(\theta_j) \prod_{s=1}^S \left[p(z_s | \pi_{z_{s-1}}) p(D_s | \omega_{z_s}) \prod_{t=t_{s-1}+1}^{t_s} p(y_t | \theta_{z_s}) \right] \end{aligned}$$

Unlike the plain HDP-HMM, the HDP-HSMM explicitly decouples duration statistics from transition dynamics, thus breaking the geometric tenure constraint inherent to Markovian state transitions (Johnson et al., 2012, Johnson et al., 2012, Wang et al., 2017).

2. Posterior Inference: Blocked Gibbs Samplers

Inference in HDP-HSMMs is based on Markov chain Monte Carlo (MCMC) techniques, notably direct-assignment and weak-limit (finite truncation) blocked Gibbs samplers, leveraging segment-based updating to accelerate mixing:

• Direct-assignment sampler:
  • Alternates updating super-state labels ($z_s$) and durations ($D_s$) using a Chinese Restaurant Franchise–style update.
  • In each iteration:
  • Update segment labels conditioned on the rest:

  $$p(z_s = k | \text{rest}) \propto (n_{z_{s-1},k}^{-s} + \alpha \beta_k) (n_{k, z_{s+1}}^{-s} + \alpha \beta_{z_{s+1}}) \mathcal{L}_k(y_{t_{s-1}+1:t_s})$$

  where $\mathcal{L}_k$ is the marginal likelihood under $F(\theta_k)$. - Re-segment given current super-state chain via backward-messages and forward sampling. - Update stick-breaking weights $\beta$, transition distributions $\pi_j$, emission and duration parameters from conjugate posteriors.

• Weak-limit approximation:
  • Employs a finite Dirichlet prior of dimension $K$ for computational tractability:

  $$\beta \sim \mathrm{Dir} \left(\frac{\gamma}{K}, \ldots, \frac{\gamma}{K}\right), \quad \pi_j \sim \mathrm{Dir}(\alpha \beta_1, \ldots, \alpha \beta_K)$$ - Blocked Gibbs steps alternate segment sampling (via semi-Markov message passing), emission/duration parameter updates, and parameter resampling.

Both approaches exploit semi-Markov message-passing (backward-filtering/forward-sampling) to efficiently handle arbitrary duration laws. The weak-limit sampler generally exhibits improved mixing performance over direct-assignment, especially for moderate $K$ (Johnson et al., 2012, Johnson et al., 2012, Taniguchi et al., 2015).

3. Explicit-Duration Modeling and Its Impact

The distinguishing feature of the HDP-HSMM is the explicit parametrization of state dwell-time distributions. Each state's segment lengths are governed by an arbitrary prior $g(\cdot|\omega_j)$, which can be Poisson, negative binomial, delayed geometric, or another suitable discrete measure. This flexibility permits modeling of non-geometric sojourns, critical for applications where HMMs and their nonparametric extensions (e.g., HDP-HMM) fail due to their implicit geometric assumption (Johnson et al., 2012, Johnson et al., 2012):

• In a standard HMM (and HDP-HMM), the probability that a state persists for $d$ consecutive observations is $\pi_{ii}^{d-1}(1 - \pi_{ii})$ (i.e., geometric).
• The HDP-HSMM allows $p(D = d \mid x = j) = g(d \mid \omega_j)$, fully decoupled from transitions.
• The model supports direct incorporation of prior knowledge about expected segment durations and enforces interpretable, application-specific dwell-time constraints.

A plausible implication is that the HDP-HSMM avoids "over-segmentation" and reduces false state switching, resulting in more semantically interpretable latent states (Johnson et al., 2012, Wang et al., 2017, Aguirre et al., 2023).

4. Empirical Results and Applications

Speaker Diarization

In speaker diarization tasks (NIST Rich Transcription data), HDP-HSMMs outperform both the sticky HDP-HMM and the plain HDP-HMM:

• HDP-HSMM achieves normalized Hamming error $\approx 0.4$ in $\approx 200$ iterations, while sticky HDP-HMM requires $5$k–$30$k iterations.
• Inferred number of speakers closely matches ground truth (within $\pm 1$ for most cases).
• The approach models non-geometric speaking turns, yielding superior interpretability (Johnson et al., 2012).

Morse-Code Pattern Discovery

For Morse code audio, the HDP-HSMM recovers all true latent classes (dot, dash, silence), correctly modeling segment lengths, whereas the HDP-HMM collapses states (tone/silence only) and cannot distinguish dots from dashes due to the geometric duration assumption (Johnson et al., 2012).

Driving Pattern Analysis

Applied to naturalistic car-following data, HDP-HSMMs uncover $\approx 75$ semantically meaningful primitive patterns per driver, supporting clustering and semantic labeling of driving behaviors. The explicit dwell-time modeling prevents spurious rapid switching, commonly observed in HDP-HMM and sticky HDP-HMM (Wang et al., 2017, Aguirre et al., 2023).

Extension to Language Acquisition

The HDP-HSMM forms a building block for hierarchical models integrating language and acoustic constraints—e.g., the Hierarchical Dirichlet Process Hidden LLM (HDP-HLM) for unsupervised discovery of words and phonemes directly from continuous speech signals (Taniguchi et al., 2015).

5. Robust Variants and Over-Splitting Mitigation

The nonparametric nature of the HDP prior can induce redundant state splitting (i.e., multiple near-identical clusters). The robust HDP-HSMM (rHDP-HSMM) addresses this by merging states with similar emission parameters within each Gibbs iteration:

• Redundant states $i, j$ are identified if $\| \theta_i - \theta_j \|_2 \leq \tau$, for a user-specified $\tau$.
• Merging involves reassigning all timepoints from the redundant set $J_i$ to a single representative $j^*$ and down-weighting the stick-breaking mass of others.
• Empirical studies show rHDP-HSMM yields fewer, more interpretable states, faster convergence, and stable parameter estimates (e.g., correctly recovering true states in $80\%$ of replicates vs. near $0\%$ for the unmodified HDP-HSMM) (Aguirre et al., 2023).

6. Comparative Perspective and Implementation Considerations

The following comparison summarizes key differences relevant to practical time-series modeling:

Model	Duration Law	State Cardinality	Transition Prior	Tendency to Over-Split?
HMM, HDP-HMM	Implicit geometric	Fixed/HDP	Categorical/HDP	Yes (HDP-HMM)
Sticky HDP-HMM	Implicit geometric (with self-bias)	HDP	Sticky HDP	Reduced
HDP-HSMM	Arbitrary explicit	HDP	HDP/Segmented	Yes (mitigated by rHDP)
rHDP-HSMM	Arbitrary explicit	HDP	HDP/Segmented	Suppressed (via merging)

Efficient inference is achieved through blocked Gibbs sampling with backward-filter/forward-sample recursions, and computational complexity per iteration is $O(T\,d_{\max}\,K + K^2)$ for the truncated weak-limit sampler. Changepoint detection and precomputing emission likelihoods further reduce computational overhead (Johnson et al., 2012, Johnson et al., 2012).

7. Extensions and Ongoing Developments

The HDP-HSMM's explicit semi-Markov structure and Bayesian nonparametric machinery provide a foundation for further model extensions, such as:

• Factorial source models for power disaggregation (Johnson et al., 2012).
• Double-articulation analyzers for direct language acquisition from continuous speech via extensions like the HDP-HLM (Taniguchi et al., 2015).
• Robust inference via explicit state-merging schemes (rHDP-HSMM) to address over-fragmentation (Aguirre et al., 2023).

The model and its inference algorithms have seen implementation in toolkits such as "pyhsmm" supporting blocked Gibbs and beam sampling, facilitating application to complex real-world structured time-series data.

References:

(Johnson et al., 2012, Johnson et al., 2012, Wang et al., 2017, Taniguchi et al., 2015, Aguirre et al., 2023)