State-Dependent Random Walks

• State-dependent random walk processes are stochastic systems where transition probabilities vary with the current state, history, or environment, unlike constant-probability Markov chains.
• They employ advanced methods such as matrix-analytic techniques, branching process decompositions, and excursion analysis to determine recurrence, tail asymptotics, and stationary measures.
• These models find applications in diverse areas like random environments, population dynamics, and stochastic optimization, uncovering complex non-Markovian behaviors with significant predictive implications.

A state-dependent random walk process is a stochastic system in which the transition probabilities, jump rates, or structure of the walk are functions of the current state, history, or environment occupied by the walker. Such models generalize classical homogeneous Markov chains, yielding rich asymptotic and recurrence behaviors that depend crucially on local and global state dependence. State-dependent random walks have been analyzed in discrete and continuous time, on lattices, strips, quadrants, and more general graphs, revealing new phenomena in recurrence, stationarity, ergodicity, and tail asymptotics.

1. Model Classes and State-Dependence Architecture

A prototypical state-dependent random walk $(X_n)$ may evolve on a subset of $\mathbb{Z}^d$ or a product space $S$. Transition probabilities are explicitly or implicitly parameterized by the current state $i$ or local environment:

• $(1,R)$-Reflecting Random Walks: For chain $(X_n)$ on $\mathbb{Z}^+=\{0,1,\ldots\}$, at $i\ge1$, transitions are $X_{n+1}=i-1$ with probability $q(i)$, $X_{n+1}=i$ with $r(i)$, $X_{n+1}=i+k$ with $p_k(i)$ for $k=1,\dots,R$; at $0$, $q(0)=0$. Transition probabilities $q(\cdot), r(\cdot), p_k(\cdot)$ are arbitrary (subject to positivity and normalization) functions of state (Hong et al., 2013).
• Half-Strip/Quadrant Models: On $S=\cup_nL_n$ with $L_n=\{(n,i):i\in\Omega\}$, transitions depend on “level” $n$ and “phase” $i$ via matrices $(Q_n, P_n, R_n)$; state dependence is indexed by level (Hong et al., 2013), or in two dimensions by $(n,m)$ and a vector of phases (O'Reilly et al., 2023).
• Random Walk in Changing Environment (RWCE): At each $t$, environment conductances $C_t:E\rightarrow[0,\infty)$ dictate step probabilities, possibly modified adaptively by the walk’s local trajectory or history (Amir et al., 2015).
• Birth-Death Dynamical Environment: Transition rates at $x$ and $t$ depend on occupation numbers accorded by an independent local birth-death process $\omega_x(t)$, and random walks, in continuous time, move with rate $\varphi(\omega_x(t))$ (Fontes et al., 2022).
• Higher-Order/Tensor-Driven (Spacey) Random Walks: Transition laws derived from order-$m$ tensors, with transitions chosen by sampling prior states with occupation probability and moving according to tensor entries; this yields explicit non-Markovian, state-dependent dynamics (Benson et al., 2016).

2. Recurrence, Positive Recurrence, and Tail Asymptotics

Positive recurrence is characterized by explicit stationary measures or Lyapunov potential techniques:

• $(1,R)$ Model: The stationary measure $\mu(i)$ is given via products of state-dependent matrices $M_j$ built from $q(j)$ and $p_k(j)$; the chain is positive recurrent iff $\sum_{i=0}^\infty\mu(i)<\infty$. The stationary distribution, when normalizable, is $\pi(i)=\mu(i)/[\sum_j\mu(j)]$ (Hong et al., 2013).
• Half-Strip/Quadrant Models: Using inhomogeneous multi-type branching structures, criteria for recurrence/positive recurrence are expressible as finiteness of key series involving matrix products of $A_n, u_n$ (Hong et al., 2013). Matrix-analytic approaches for LD-QBD transforms yield equivalent global balance equations for stationary vectors (O'Reilly et al., 2023).
• RWCE: Recurrence/transience dichotomies follow from monotonicity and comparison to limiting conductance profiles, using martingale potentials. In the adaptive case, phenomena such as transience can occur even for walks bounded above by recurrent conductances, due to local state-dependent biases (Amir et al., 2015).
• Random Walk with Hyperbolic Probabilities: Site-dependent transition probabilities interpolate between symmetric and biased walks. Transience or recurrence is linked to the parameter $\xi$; for $\xi\ne0$ the walk is transient with probability of return to origin less than $1$. No stationary distribution exists unless $\xi=0$, but time-averaged mean-squared displacement shows self-averaging (Montero, 2019).

Tail asymptotics are determined by the near-critical decay of local drift or transition probabilities:

• For $(1,R)$ models in the vicinity of null-recurrence, perturbations $\varepsilon_i$ in the rates result in trichotomies: slower decay $\varepsilon_i\sim C i^{-\alpha},\,0<\alpha\le1$ can restore positive recurrence, with stationary tails $\log\pi(i)\sim -C\kappa i^{1-\alpha}$ or $-C\kappa\log i$ (Hong et al., 2013).
• In strip models, spectral radii $\rho(A)$ of limiting matrix products encode exponential tail decay rates for stationary distributions $\pi_{n,i}\sim\rho(A)^n$ (Hong et al., 2013).

3. Branching Structures and Matrix-Analytic Techniques

Intrinsic multi-type branching processes underpin rigorous analysis:

• Forward and backward excursions, hitting times, and offspring distributions yield explicit recursive equations for expected visits or transition structure (Hong et al., 2013, Hong et al., 2013).
• Mapping to LD-QBD (Level-Dependent Quasi Birth-and-Death) processes facilitates standard matrix-analytic solution techniques, including computing invariant vectors and first-passage transforms (O'Reilly et al., 2023).
• Matrix products $M_1 M_2 \cdots M_{i-1}$ or products of $A_n$ approximate occupation statistics or stationary measures (Hong et al., 2013, Hong et al., 2013).
• Discrete analogues of diagonalization theorems (Levinson–Benzaid–Lutz) control asymptotics in inhomogeneous recursion.

4. Dynamical Random Environments and Non-Markovian Structure

State-dependent random walks in dynamic or random environments, where the local rates or transition laws evolve in time and/or depend on other evolving processes, produce qualitatively new limit laws:

• Random walks governed by environment processes (Poisson fields, birth-death systems) admit strong law of large numbers and functional CLT, with explicit formulae for velocity and diffusivity derived using regeneration times and renewal theory (Hilário et al., 2014, Fontes et al., 2022).
• In environments with vanishing jump rates ($\varphi$ decreasing with occupation), subadditive ergodic techniques and stochastic domination are necessary to prove LLN/CLT, avoiding assumptions of uniform ellipticity (Fontes et al., 2022).
• Reinforcement models (spacey random walk) use occupation history to modulate transitions, converging to tensor eigenvectors; uniqueness and existence are governed by spectral positivity and contraction parameters, with polynomial fixed-point equations governing stationary distributions (Benson et al., 2016).

5. Examples, Applications, and Open Problems

State-dependent walks underpin models in reinforced/random environments, population dynamics, and stochastic optimization:

• Reinforced walks, self-avoiding walks, and loop-erased walks manifest as state-dependent (often adaptive) RWCEs, with recurrence/transience dichotomies dependent on path-history (Amir et al., 2015).
• In "stairs" models, divergence and recurrence are governed by sequences $\{a_n\}$ controlling drift; explicit phase-based constructions prove divergence with any prescribed probability $\sigma<1$, but almost-sure divergence remains open (Li et al., 2018).
• The spacey random walk captures higher-order dependencies, matching predictive accuracy of second-order Markov models for structured trajectory data, while maintaining first-order asymptotics (Benson et al., 2016).
• Models incorporating geometrically motivated transition rates (hyperbolic probabilities) have implications for diffusion in heterogeneous media, internet routing, and migration in fitness landscapes, with unique ergodic breakdown phenomena (Montero, 2019).

6. Summary of Analytical Tools and Limitations

State-dependent random walks demand analytical frameworks extending beyond standard matrix-analytic Markov chain theory:

• Branching-process decomposition and excursion analysis provide explicit recurrence/occupation formulas (Hong et al., 2013, Hong et al., 2013).
• Matrix-analytic methods (rate matrices, balance equations, spectral theory) are indispensable for LD-QBD representations (O'Reilly et al., 2023).
• Subadditive ergodic theory, stochastic domination, and coupling arguments are required when rates may vanish or environments are unbounded (Fontes et al., 2022, Li et al., 2018).
• Open problems persist in establishing sharp recurrence/transience boundaries for adaptive, non-uniformly elliptic, or non-Markovian state dependence, and in quantifying global stability for stochastic nonlinear fixed-point equations in tensor-driven walks.