Finite-Alphabet Markov Chains

• Finite-alphabet Markov chains are discrete stochastic processes defined on a finite state space via a transition probability matrix for modeling categorical systems.
• They exhibit rich dynamical properties including Poincaré and Devaney chaos, where positive transitions ensure transitivity, dense periodic points, and sensitive dependence on initial conditions.
• Structural tools such as skeleton orders, context equivalence, and large deviation principles enable efficient parameter reduction and model selection in high-dimensional settings.

A finite-alphabet Markov chain is a stochastic process whose state space is a finite set and whose transitions are governed by a matrix of probabilities. These chains serve as foundational models for discrete-time, discrete-state systems across probability, information theory, statistical mechanics, and ergodic theory. Remarkably, such chains possess intrinsically rich structures: their dynamical, statistical, and large deviation properties connect to deterministic chaos, information-theoretic dimension, entropy, combinatorial representation, and computational complexity.

1. Formal Structure and Dynamics

Let $S = \{s_1, \ldots, s_m\}$ be a finite set equipped with a metric $d(\cdot,\cdot)$. A time-homogeneous Markov chain on $S$ is specified by an $m \times m$ transition matrix $P = (p_{ij})$, where $p_{ij} \geq 0$ and $\sum_{j=1}^m p_{ij} = 1$. The one-step dynamic is

$\Pr\{X_{n+1} = s_j \mid X_n = s_i\} = p_{ij}.$

Alternatively, the process may be expressed as $X_n = f(X_{n-1}, Y_n)$, where $Y_n$ are i.i.d. noise variables and $f$ is deterministic so that $\Pr\{f(s_i, Y) = s_j\} = p_{ij}$ (Akhmet, 2020).

A realization $(X_0, X_1, X_2, ...)$ is often encoded as a symbolic sequence $F = i_0i_1i_2...$: permitted transitions satisfy $p_{i_k,i_{k+1}}>0$. The symbolic space $\mathcal{F}$ is the set of all such sequences, equipped with the ultrametric

$$\delta(F,G) = \sum_{k=0}^{\infty} \frac{d(s_{i_k}, s_{j_k})}{2^k},$$

for $F = i_0i_1...$, $G = j_0j_1...$.

2. Classification, Reduction, and Structural Invariants

Higher-order Markov chains (order $m \geq 1$) generalize the first-order case: the conditional probability kernel $p:A^m \times A \rightarrow [0,1]$ defines, for each context $x \in A^m$, the next-symbol law. These can be recast as first-order chains on $A^m$, but the resulting transition matrix is typically sparse and encodes combinatorial constraints inaccessible to brute-force analysis.

The "skeleton" of a transition kernel [Editor’s term], introduced in (Gallesco et al., 10 Jan 2026), is a minimal object that records intrinsic patterns of forbidden transitions. For each $x \in A^m, a \in A$, define

$$\tau(x, a) = \min\left\{ \begin{array}{l} i \geq 0:\,\,\bigl[\forall y\in A^{i-1},\,\,p(y\,x_{m-i+1}^m, a) = 0\bigr] \ \text{or}\ \bigl[\forall y\in A^{i-1},\,\,p(y\,x_{m-i+1}^m, a) > 0\bigr] \end{array} \right\}.$$

The maximal $\tau_x := \sup_{a \in A} \tau(x, a)$ over all contexts gives the skeleton order $K := \sup_{x \in A^m} \tau_x$.

The skeleton is encoded as a binary $\lvert A\rvert^K \times \lvert A\rvert^K$ matrix $\mathbb{M}$, with standard notions of closed classes, periods, recurrence, and transient behavior now computable at cost $\mathcal{O}(\lvert A\rvert^{K})$ (Gallesco et al., 10 Jan 2026).

3. Dynamical and Chaotic Properties

Finite-alphabet Markov chains are Poincaré chaotic: for strictly positive $p_{ij}$, every realization is a segment of a unique unpredictable orbit under the shift map $\phi:\mathcal{F} \rightarrow \mathcal{F}$, $\phi(i_0i_1i_2...) = i_1i_2...$ (Akhmet, 2020).

Theorem (Devaney chaos): If $p_{ij} > 0$ for all $i,j$, then $(\mathcal{F}, \phi)$ is transitive, has dense periodic points, and is sensitive to initial conditions. Thus, $\phi$ is chaotic in the strict sense.

Theorem (Poincaré chaos): Under the same conditions, $\phi$ admits at least one unpredictable point $F^* \in \mathcal{F}$ whose orbit is Poincaré chaotic (closure is quasi-minimal, no proper closed invariant subsets).

Consequences: Every finite block observed in any path is a contiguous segment of an unpredictable orbit with probability one, and sample path behavior manifests both recurrence and sensitive divergence (Akhmet, 2020).

4. Entropy, Dimension, and Information-Theoretic Invariants

The topological entropy of the shift map on $m$ symbols is $h_{\mathrm{top}}(\phi) = \ln m$, reduced to $\ln \lambda$, where $\lambda$ is the Perron–Frobenius eigenvalue of the adjacency matrix $A_{ij} = 1_{p_{ij} > 0}$ in the case of forbidden transitions.

The Kolmogorov–Sinai (measure-theoretic) entropy is given by

$h_\mu(\phi) = -\sum_{i,j} \pi_i p_{ij} \ln p_{ij},$

where $\pi$ is the stationary distribution.

Finite-state dimension quantifies the informational content as perceived by finite automata (Bienvenu et al., 21 Oct 2025): $$\dim_{FS}(S) = \inf_{M}\limsup_{n\to\infty} \frac{D_{KL}(P_n(\Sigma\mid Q)\|\pi_M(\Sigma\mid Q))}{\log|Σ|},$$ where $D_{KL}$ is the conditional KL divergence between empirical symbol-state frequencies $P_n(b|q)$ and stationary distributions, simulations performed via finite-state irreducible Markov chains.

For high-order Markov chains (memory $N$), the Shannon entropy rate admits a bilinear approximation: $$h \simeq h_0 - \frac{1}{2\ln2}\sum_{r=1}^N \sum_{\alpha, \beta \in A} \frac{[C_{\beta\alpha}(r)]^2}{p_\alpha p_\beta} - \frac{1}{\ln2}\sum_{r_1 < r_2} \sum_{\alpha,\beta,\gamma} \frac{C_{\beta\gamma\alpha}(r_2, r_1)}{p_\alpha p_\beta p_\gamma},$$ with $C_{ij}(r)$ and $C_{ijk}(r_2, r_1)$ empirical correlators (Melnik et al., 2017).

5. Minimal Model Selection and Parameter Reduction

Not all length-$M$ histories carry distinct predictive relevance. The minimal Markov model (Gonzalez-Lopez, 2010) aggregates contexts: $s \sim s'$ iff $\forall a\in A, P(a|s)=P(a|s')$, forming equivalence classes $\mathcal{C}=S/\sim=\{L_1,\ldots,L_K\}$.

This model requires only $K(\lvert A\rvert-1)$ parameters, a dramatic reduction from the full $|A|^M(\lvert A\rvert-1)$ possibilities, and includes as special cases variable-length Markov chains (VLMC) and context-tree models. Consistent model selection is achieved via minimization of the Bayesian information criterion: $$\mathrm{BIC}(\mathcal{L}; x_1^n) = \ell_n(\mathcal{L}) - (\lvert A\rvert-1)K \frac{1}{2} \ln n,$$ where $\ell_n(\mathcal{L})$ is the log-likelihood and $K$ the number of categories (Gonzalez-Lopez, 2010).

6. Large Deviations and Empirical Frequencies

For irreducible, stationary finite-alphabet chains, the large deviation principle (LDP) for empirical pair (doublet) frequencies is determined by the conditional relative entropy (Vidyasagar, 2013): $$I(q) = \sum_{i,j}q_{ij} \ln \frac{q_{ij}}{q_i P_{ij}},$$ where $q_{ij}$ is the empirical proportion of $(i,j)$ transitions and $q_i=\sum_j q_{ij}$. The probability of observing a type class decays asymptotically as

$$\Pr\{\hat\nu^{(N)} \approx q\} \asymp \exp\left(-N I(q)\right).$$

7. Combinatorial and Representation Theory Connections

For Markov chains on a totally ordered finite alphabet, the limiting shape of RSK Young diagrams associated with random words governed by the chain reveals deep connections to random matrix theory. Specifically, the scaled row-lengths converge as centered Brownian functionals, and under certain spectral constraints (cyclic or reversible transition matrices), the asymptotic law matches the spectrum of traceless GUE matrices (Houdré et al., 2011). Cyclic Markov chains may or may not satisfy uniformity depending on the dimension and eigenvalue criteria.

8. Sequential Construction and Memory Function Decomposition

Conditional probability functions of high-order Markov chains can be decomposed into a sum of multi-linear memory function monomials. Under stationarity and ergodicity, the chain's CPF is

$$P(a_i = \alpha | a_{i-N}^{i-1}) = \sum_{k=0}^N Q^{(k)}(a_i = \alpha | a_{i-N}^{i-1}),$$

with explicit formulae for memory functions $F$ in terms of empirical stationary $k+1$-point correlations $C$, enabling efficient sequential generation of artificial sequences matching prescribed statistical properties (Melnik et al., 2017).

9. Practical Applications and Markovian Statistics

Finite-alphabet Markov chains are central in modeling categorical time series, statistical estimation, text-generation processes, and motif detection problems (e.g., gambler's ruin variations where occurrences of two patterns in Markov-generated text determine scoring processes, analyzed by embedding the process in auxiliary chains and deriving explicit probability and mean waiting time formulas) (Chi et al., 1 Jun 2025).

---

Overall, finite-alphabet Markov chains unify discrete stochastic processes, deterministic chaos, and symbolic dynamics. Their mathematical structure translates between probabilistic laws, spectral theory, and computational inference, supporting both deep theoretical results and efficient algorithms for high-dimensional, higher-order, and context-sensitive situations.