Parameterized Markov Chain Kernel

• Parameterized Markov chain kernels are families of smoothly tuned transition probabilities that enable systematic optimization of MCMC dynamics.
• They leverage exponential-family formulations, path entropy constraints, and information-geometric structures to enhance statistical estimation and reduce rejection rates.
• These kernels facilitate the design of adaptive algorithms and graph-based proposals to improve sampling performance in high-dimensional probabilistic models.

A parametrised Markov chain kernel is a collection of Markov transition kernels constructed to depend smoothly on a set of continuous or discrete parameters, enabling systematic tuning or optimization of the chain’s statistical and dynamical properties. Such parameterizations are essential for both statistical inference (estimation) and algorithm design in Markov chain Monte Carlo (MCMC), dimensionality reduction, and information geometry. Fundamental examples include exponential-family parameterizations, path-entropy–constrained kernels, one-parameter rejection-control kernels, and graph-based parameterized proposals.

1. Exponential-Family Parameterization of Markov Kernels

Given a finite state space $\mathcal X$, let $W_0(x' \to x)$ be an irreducible base Markov kernel and fix a collection of generator functions $\{F_i(x, x')\}_{i=1}^d$. For each parameter vector $$\theta = (\theta^1, ..., \theta^d) \in \Theta \subseteq \mathbb R^d$$, the unnormalized kernel is defined as

$$\bar W_\theta(x|x') := W_0(x|x') \exp\left( \sum_{i=1}^d \theta^i F_i(x,x') \right).$$

By the Perron–Frobenius theorem, $\bar W_\theta$ admits a unique maximal eigenvalue $\lambda(\theta) > 0$ and strictly positive right eigenvector $R_\theta$. Let

$$\psi(\theta) := \ln \lambda(\theta), \qquad h(x,x') := W_0(x|x'),$$

the normalized, stochastic transition kernel is then

$$P_\theta(x|x') = \exp\left( \sum_{i=1}^d \theta^i F_i(x,x') - \psi(\theta) \right) h(x, x').$$

Here, the $F_i$ act as sufficient statistics. The function $\psi(\theta)$ is the log-partition (potential) function as in the classical exponential family, generalizing familiar constructions from statistical estimation to Markov kernels (Hayashi et al., 2014).

2. Statistical Estimation: Likelihood, Score, and Fisher Information

When observing a trajectory $X_1, ..., X_{n+1}$ from the chain with kernel $P_\theta$:

• The log-likelihood is

$$L(\theta) = \sum_{t=1}^n \left[ \sum_{i=1}^d \theta^i F_i(X_{t+1}, X_t) - \psi(\theta) \right] + \text{const}.$$

• The score function is

$$S_i(\theta) = \frac{\partial L(\theta)}{\partial \theta^i} = \sum_{t=1}^n F_i(X_{t+1}, X_t) - n \frac{\partial \psi}{\partial \theta^i}(\theta).$$

• The Fisher information matrix is

$$I_{ij}(\theta) = n\, \frac{\partial^2 \psi(\theta)}{\partial \theta^i \partial \theta^j}.$$

Under ergodicity assumptions, the sample-mean estimator for the expectation parameters $\eta_i(\theta) := \mathbb E_\theta[F_i(X', X)]$,

$$\hat\eta_i = \frac{1}{n} \sum_{t=1}^n F_i(X_{t+1}, X_t)$$

is unbiased and asymptotically efficient, achieving the Cramér–Rao lower bound: $$\mathrm{Var}_\theta(\hat\eta) = \frac{1}{n} \nabla^2 \psi(\theta) + o(1/n).$$ This sample mean is thus an optimal estimator for the expectation parameters in the exponential family setting (Hayashi et al., 2014).

3. Information-Geometric Structure

The space of Markov kernels on $\mathcal X$ forms a convex subset of $\mathbb R^{|\mathcal X|^2}$, endowed with a natural information geometry:

• e-connection: The exponential family $\{P_\theta\}$ is e-flat (zero e-curvature), with $\theta$-coordinates affine under the exponential connection.
• m-connection: The dual affine structure is determined by $\eta$, the vector of expectation parameters; these are affine under the mixture (m-) connection.
• Dual coordinates: $$\eta_i(\theta) = \frac{\partial \psi}{\partial \theta^i}(\theta)$$.
• The exponential family is thus a dually flat submanifold, and the normalized generator functions $F_i$ provide a sufficient-statistics representation (Hayashi et al., 2014).

4. Parameterized Kernels via Path Entropy Optimization

Beyond the exponential family, general parameterized Markov kernels arise by maximizing path entropy subject to constraints. Given a symmetric affinity kernel $k(i, j) > 0$ and optional constraints on stationary measures and path-wise averages (e.g., cost, distance), the path entropy

$\mathcal S = -\sum_{i, j} \pi_i q_{ij} \ln q_{ij}$

is maximized with respect to $q_{ij}$ (the transition matrix) and possibly $\pi_i$ (the stationary distribution) (Dixit, 2018). Imposing dynamical constraints of the form

$$\sum_{i,j} \pi_i q_{ij} r^{(k)}_{ij} = \overline r^{(k)}$$

is achieved through Lagrange multipliers $\lambda_k$, yielding kernels of the form

$$q_{ij} = \alpha\, \rho_i \rho_j\, \exp\left( -\sum_{k} \lambda_k r^{(k)}_{ij} \right)/\pi_i.$$

Adjusting $\lambda_k$ continuously tunes the family, enabling user-prescribed stationary and dynamical features. For the maximum-entropy random walk (MERW), when $\pi_i$ is not fixed, the kernel takes the form

$$q_{ij} = \frac{1}{\Lambda} \frac{\psi_j}{\psi_i} k(i, j), \qquad \pi_i \propto \psi_i^2$$

where $\psi$ is the leading eigenvector of $k(i,j)$ (Dixit, 2018).

5. One-Parameter Rejection-Control Kernels and MCMC Efficiency

In MCMC, the choice of the Markov kernel critically affects sampling efficiency. One important parameterized family is the rejection-control kernel defined for discrete local updates. Let $\pi_i$ be local weights, $F_i = \sum_{k=1}^i \pi_k$, and introduce a "shift" parameter $s \in (0, F_n)$ (or $\alpha = s/F_n \in (0, 1)$ if normalized). The kernel is constructed by forming the flows

$$v_{ij} = \max\big(0, \min(\Delta_{ij}, \pi_i + \pi_j - \Delta_{ij}, \pi_i, \pi_j)\big) +\max\big(0, \min(\Delta_{ij} - F_n, \pi_i + \pi_j + F_n - \Delta_{ij}, \pi_i, \pi_j)\big)$$

with $\Delta_{ij} = F_i - F_{j-1} + s$, and transition probability $P_{ij} = v_{ij}/\pi_j$ (Suwa, 2022).

Tuning $s$ affects the probability of rejection and the autocorrelation time $\tau_{\mathrm{int}}$:

• With sequential updates, $$\tau_{\mathrm{int}} \approx a \exp(-b\,p_{\mathrm{rej}})$$ where $p_{\mathrm{rej}} = \sum_j P_{jj}$.
• With random updates, $$\tau_{\mathrm{int}} \approx \alpha (1 - p_{\mathrm{rej}})^{-\gamma}$$.

Choosing $s = F_n/2$ yields a reversible kernel that minimizes rejection, universally optimizing $\tau_{\mathrm{int}}$ over various discrete-variable models. This kernel framework unifies and generalizes commonly used kernels such as Metropolis–Hastings, heat-bath, Metropolized Gibbs, and the Suwa–Todo algorithm (Suwa, 2022).

6. Graph-Based Parameterized Kernels for MCMC Acceleration

In high-dimensional Bayesian computation, a graph-parameterized kernel can be constructed using approximate samples. For a set of nodes $$S = \{s_1, ..., s_m\} \subset \Theta \subseteq \mathbb R^d$$, one forms a directed graph $G$ with edges $(i \to j)$ weighted by $w_{ij} \geq 0$, leading to a proposal distribution $q(i \to j) = w_{ij}/\sum_k w_{ik}$. Metropolis–Hastings corrections restore invariance to the true posterior: $$\alpha(s_i, s_j) = \min\left\{1, \frac{\pi(s_j)\, q(j \to i)}{\pi(s_i)\, q(i \to j)} \right\}.$$ Weight optimization may maximize the empirical expected squared jumped distance (ESJD)

$$\max_{w_{ij} \ge 0} \sum_i \sum_{j} w_{ij}\, \alpha(s_i, s_j)\, \|s_i - s_j\|^2, \quad \sum_j w_{ij} = 1$$

or minimize a penalty involving log-density differences, distances, and entropy regularization. Embedding this graph kernel as a mixture with a local baseline kernel (e.g., random-walk MH, Gibbs) produces a family of MCMC samplers whose mixing time improves strictly if the ergodic flow across bottlenecks is increased. The approach generalizes to continuous parameterizations (basis function proposals, normalizing flows) and is scalable via sparsified or pruned graphs (Duan et al., 2024).

7. Curved Exponential Families and Information-Geometric Projections

A curved exponential family is defined by restricting $\theta$ to a lower-dimensional manifold $\theta = \theta(\xi)$, $\xi \in \mathbb R^{d'}$ with $d' < d$. In this context, the Markov chain version of the Pythagorean theorem holds: $$D(P \| P_{\theta(\xi'')}) = D(P \| P_{\theta(\tilde \xi)}) + D(P_{\theta(\tilde \xi)} \| P_{\theta(\xi'')}),$$ with $D(\cdot\|\cdot)$ the Kullback–Leibler divergence under the stationary joint law. The estimator

$$\hat\xi = \arg\min_{\xi} D(P_{\hat\eta} \| P_{\theta(\xi)})$$

is asymptotically efficient, with covariance attaining the curved-family Cramér–Rao bound: $$\mathrm{Cov}(\hat\xi) = \frac{1}{n}\left[ A^T I(\theta)^{-1} A \right]^{-1} + o(1/n), \qquad A_{ij} = \frac{\partial \eta_i}{\partial \xi_j}|_{\xi = \xi_0}.$$ This structure allows for statistically optimal estimation and systematic geometric interpretations of constraint-manifold models (Hayashi et al., 2014).