Lyapunov Drift: Stability & Convergence

• Lyapunov drift condition is a tool that uses a Lyapunov function to ensure system stability by enforcing a negative expected increment outside a specified region.
• It applies across diverse fields such as Markov chains, SDEs, queueing networks, reinforcement learning, and linear systems, providing explicit rates of convergence and stability guarantees.
• Generalizations like random-time drift and drift-plus-penalty methods extend these techniques to numerical analysis and control optimization, enabling robust regularity and performance assessments.

The Lyapunov drift condition is a fundamental analytic tool for certifying and quantifying stability, convergence rates, and regularity properties in stochastic, deterministic, and control-theoretic models. Its central role spans Markov chains, stochastic differential equations (SDEs), queueing networks, delay equations, distributed systems, and reinforcement learning, unifying disparate approaches to stability and performance analysis under a common framework. At its core, the Lyapunov drift condition asserts that a suitable function $V$—the "Lyapunov function"—exhibits dissipative behavior under the evolution defined by a system's dynamics: its expected increment (or generator action) is negative outside a prescribed small or recurrent region, thereby forcing recurrence, finite moments, or exponential convergence.

1. Canonical Formulation and Principles

The classical Lyapunov drift condition asserts that for a discrete-time Markov chain $(X_n)$ on a Polish space $X$, or for a continuous-time process with generator $L$, there exists a measurable function $V:X\to[1,+\infty)$ (norm-like, with $V(x)\to\infty$ as $|x|\to\infty$), a small (petite) set $C\subset X$, constants $\lambda\in(0,1)$, $b<\infty$, and minorization measure $\nu$ with weight $\alpha>0$, so that:

$$P\,V(x) \le (1-\lambda)\,V(x) + b\,\mathbf{1}_C(x),\qquad P\,\mathbf{1}_A(x) \ge \alpha\nu(A)\mathbf{1}_C(x)\quad\forall A\subset X$$

(Taghvaei et al., 2020).

For SDEs, generator inequalities take the form:

$(\partial_t + L)V(t,x) \le C V(t,x)$

(Chak, 2022),

or, in the presence of singular or distribution-dependent coefficients, more generally:

$$L V(x, \mu) \le \zeta\big(1 + \mu(\mathbf V) + \mathbf V(x) \big)$$

(Noelck, 2024).

For queueing systems:

$\Delta V(t) \le B - \epsilon\sum_{n} Q_n(t)$

(Neely, 2010, Xu et al., 4 Jun 2025).

These conditions are designed to provide uniform moment bounds, finite recurrence times, and often explicit rates of convergence to equilibrium.

2. Random-Time and State-Dependent Drift Generalizations

Standard one-step drift criteria are generalized in stochastic networks and control by sampling the Lyapunov function at random, possibly state-dependent stopping times $(\tau_n)$, resulting in "random-time Lyapunov drift." For such cases:

$$E[V(X_{\tau_{n+1}}) | \mathcal F_{\tau_n}] \le V(X_{\tau_n}) - \delta(X_{\tau_n}) + b\,\mathbf{1}_C(X_{\tau_n}),$$

with complementary cost accumulation:

$$E\Big[\sum_{t=\tau_n}^{\tau_{n+1}-1} f(X_t) r(t-\tau_n) \Big| \mathcal F_{\tau_n}\Big] \le \delta(X_{\tau_n})$$

(Yüksel et al., 2010, Zurkowski et al., 2013).

These allow verification at non-uniform, event-triggered intervals, critical in networked control and MCMC algorithms. Ergodicity follows under appropriate distributional assumptions on the inter-sampling times; geometric and subgeometric rates are available (Zurkowski et al., 2013).

3. Lyapunov Drift for Regularity and Numerical Analysis of SDEs

Lyapunov drift conditions have supplanted global Lipschitz hypotheses for regularity in Kolmogorov equations and numerical schemes for SDEs:

• Local Lipschitz via Lyapunov: Coefficient growth is allowed to depend on $G(t,x)$, with $G=o(\log V)$, $G=o(\sqrt{\log V})$, ensuring moment integrability over finite time horizons (Chak, 2022).
• Regularity of Semigroups: Under $(\partial_t + L)V \le C V$, differentiability of the associated Markov semigroup and existence of classical PDE solutions are established.
• Lyapunov-Tamed Schemes: For SDEs with singular drifts, drift coefficients are truncated wherever $V(x)$ or $|b(x)|$ is large. The error analysis leverages uniform moment bounds from the Lyapunov drift to obtain classical $L^p$ rates for Euler-type methods—robust even in the presence of singularities (Johnston et al., 23 Jan 2026).

A tabulated summary:

Setting	Drift Form	Result Type
SDEs/Kolmogorov PDEs	$(\partial_t + L)V \le C V$	Regularity, classical solution
Stochastic Euler scheme	$$\mathcal L(V^p) + \frac{1}{q_p} |\nabla V^p|^{q_p} \le a_p + b_p V^p$$	Scheme convergence
Queueing networks	$\Delta V \le B - \epsilon \sum Q$	Rate stability, backlog bounds

4. Queue Stability and Drift-Plus-Penalty Optimization

In stochastic queueing networks and reinforcement learning, quadratic Lyapunov drift and drift-plus-penalty inequalities formalize the trade-off between stability and performance objectives (Neely, 2010, Xu et al., 4 Jun 2025):

• The quadratic Lyapunov $V(Q) = \frac{1}{2}\sum_k Q_k^2$ is used.
• Drift-plus-penalty condition:

$$\Delta V(t) + V\,\mathbb{E}[p(t) | Q(t)] \le B - \epsilon \sum_k Q_k(t)$$

yields time-average expectation bounds and, with additional fourth-moment conditions, almost sure rate stability and backlog boundedness. Model-free RL policies can be designed to optimize instantaneous Lyapunov drift-plus-penalty, which ensures queue stability and near-optimality (Xu et al., 4 Jun 2025).

5. Foster–Lyapunov Criterion, Spectral Gap, and Functional Inequalities

In reversible Markov chains, the Foster–Lyapunov drift condition tightly links stochastic stability to spectral properties and functional inequalities (Taghvaei et al., 2020):

• Drift condition:

$P V(x) \le (1-\lambda) V(x) + b\, \mathbf{1}_K(x)$

plus minorization on a small set $K$.

These yield Poincaré inequalities:

$$\|f\|_{2,\pi}^2 \le \frac{1}{\beta_+} \langle f, (I-P)f \rangle_\pi, \quad \beta_+ = \frac{\lambda}{1 + 2b/\alpha}$$

which imply explicit spectral gap bounds for convergence rates of Markov processes. Non-reversible extensions rely on applying the drift to $P^\dagger P$ or $P^2$.

6. Lyapunov Drift in Structured Linear Systems

For deterministic linear systems $\dot x = A x$, stability is often certified via Lyapunov inequalities:

$A^T P + P A \prec 0$

where block-diagonal or diagonal $P$ are desirable in large-scale distributed systems. Existence of such certificates can be determined via $\mathcal{H}_+$-matrix theory and small-gain conditions (Sootla et al., 2016):

• If $-A$ is an $\mathcal{H}_+$-matrix and the comparison matrix is nonsingular, explicit diagonal $P$ can be constructed.
• Block partitions can be handled via transfer matrix small-gain conditions and associated Riccati equations.

7. Lyapunov Drift for (Non-)Strong Ergodicity

The classical Lyapunov drift condition characterizes strong (uniform exponential) ergodicity via:

$L V(x) \le -\phi(V(x)) + b\, \mathbf{1}_C(x)$

with $\phi(r) = \lambda r$ (exponential), $\phi(r)= r^\alpha$ or $r / (\log r)^{\beta}$ (subgeometric) (Mao et al., 2019). Mao–Wang [2019] provide a dual criterion for non-strong ergodicity: pairs of norm-like functions $u, v$ satisfy $L u \ge -1$ and $L v \le d \mathbf{1}_H$, with $u/v \to 0$ outside large sets, implying failure of strong ergodicity. Examples include diffusions on manifolds and Ornstein–Uhlenbeck processes with symmetric $\alpha$-stable noise; here, strong ergodicity is determined independently of $\alpha$.

Conclusion

The Lyapunov drift condition is a keystone analytic method facilitating rigorous stability, ergodicity, regularity, and algorithmic performance guarantees across stochastic processes, control systems, and complex optimization domains. Its extensive generalizations—random-time drift, state-dependent intervals, measure-dependent forms, and block-structured linear inequalities—ensure applicability to current models in mathematical control, probability, numerical analysis, and queueing theory.