Propagation of Chaos in Rényi Divergences

• The paper establishes a sharp O(d q²/N²) convergence rate for propagation of chaos using log-Sobolev inequalities and a Donsker–Varadhan splitting approach.
• It extends the entropic framework beyond KL divergence by rigorously characterizing fluctuations and concentration phenomena in weakly interacting diffusions.
• The methodology relies on strong isoperimetry and weak interaction assumptions, with illustrative Gaussian examples confirming the optimality of the derived scaling laws.

Propagation of chaos in Rényi divergences quantifies the extent to which finite-particle approximations to mean-field interacting diffusion systems decorrelate as the system size $N$ grows, using the $q$-Rényi divergence as a measure of marginal independence. Recent work establishes sharp $O(d q^2/N^2)$ rates for the stationary measures of weakly interacting diffusions, extending the entropic framework previously developed for Kullback–Leibler (KL) divergence. This approach rigorously characterizes fluctuations and concentration phenomena in high-dimensional and high-particle-number regimes, under conditions of strong isoperimetry and weak particle interaction (Zhang, 15 Jan 2026).

1. $q$-Rényi Divergence and Its Role

For probability measures $\mu,\nu$ on $\mathbb{R}^d$ with $\mu\ll\nu$ and real $q > 1$, the $q$-Rényi divergence is defined as

$$\mathsf R_q(\mu\Vert\nu) = \frac{1}{q-1} \log\Bigl( \mathbb E_\nu\Big[(\frac{d\mu}{d\nu})^q \Big] \Bigr).$$

Setting $\rho = d\mu/d\nu$, this simplifies to

$$\mathsf R_q(\mu\Vert\nu) = \frac{1}{q-1} \log(\nu[\rho^q]), \quad \lim_{q \downarrow 1} \mathsf R_q(\mu\Vert\nu) = \mathrm{KL}(\mu\Vert\nu).$$

Rényi divergences interpolate between various risk-sensitive divergences and play a central role in non-asymptotic information-theoretic analysis. In the context of interacting diffusions, propagation of chaos in Rényi divergence precisely captures the rate at which empirical marginals approach statistical independence as $N \to \infty$.

2. Main Theorem: Sharp Rényi Propagation of Chaos Rate

For an $N$-particle interacting diffusion system at stationarity, let $\mu^{[k]}$ denote the $k$-marginal of the stationary law and $\pi$ the mean-field Gibbs minimizer. The main result asserts that, under specified regularity and weak interaction assumptions, there exist constants $C,c > 0$—independent of $d,N,q$—such that

$$N \ge \widetilde\Omega(\sqrt{d}\, q) \implies \mathsf R_q(\mu^1 \Vert \pi) \le C\, \widetilde O\left(\frac{d q^2}{N^2}\right),$$

where hidden constants depend only on the log-Sobolev constant $C_{\rm LSI}$ and interaction smoothness parameter $\beta_W$, and $\widetilde O$ notation suppresses polylogarithmic terms. The result establishes an optimal $O(d q^2/N^2)$ rate for first marginals; the arguments also yield bounds for general $k$-marginals of the form $O(d k^3 q^2/N^2)$ under analogous conditions.

3. Technical Conditions and Setting

The result holds for stationary solutions of interacting diffusion systems characterized by:

• Potentials and Interactions: Confinement potential $V:\mathbb R^d \to \mathbb R$ and interaction kernel $W:\mathbb R^d \to \mathbb R$, with mean-field Gibbs law $\pi \propto \exp(-V - W*\pi)$.
• Assumption 2.1 (Smoothness): $\|\nabla W(x) - \nabla W(y)\| \le \beta_W\|x-y\|$ for all $x,y \in \mathbb R^d$ (interaction gradient Lipschitz with parameter $\beta_W$).
• Assumption 2.2 (Uniform log-Sobolev/Isoperimetry): There exists $\bar C_{\rm LSI} < \infty$ such that for all $f \in C_c^\infty$,

$$\mathrm{Ent}_\pi(f) \le \bar C_{\rm LSI} \mathbb E_\pi[\|\nabla f\|^2],$$

and this holds uniformly for all conditional measures of $\mu^{1:N}$.

• Assumption 2.3 (Very Weak Interaction): $\beta_W \bar C_{\rm LSI} \ll 1$; i.e., the interaction is a small perturbation on the log-Sobolev scale.

These assumptions guarantee uniqueness of the relevant Gibbs measures and well-posedness of both McKean–Vlasov and $N$-particle stationary dynamics.

4. Proof Structure and Key Components

The analysis proceeds through several layers:

1. LSI-to-Rényi Inequality: For any $\nu$ satisfying a log-Sobolev inequality, Lemma 3.1 shows

$$\mathsf R_q(\mu\Vert\nu) \le \frac{q\,C_{\rm LSI}}{2} \mathsf{RF\!I}_q(\mu\Vert\nu),$$

where the Rényi–Fisher information is

$$\mathsf{RF\!I}_q(\mu\Vert\nu) = q\, \mathbb E_\mu\Bigl[\psi^q\,\|\nabla\log\rho\|^2\Bigr], \quad \psi^q \propto \rho^{q-1}.$$

2. Donsker–Varadhan Splitting: The Rényi–Fisher information can be decomposed as

$\mathsf{RF\!I}_q \le A+B+C,$

with

$$\begin{array}{rl} A & = \mathrm{FI}(\mu\Vert\nu) \ B & = \mathrm{KL}(\widetilde P \Vert \mu) \ C & = \log \mathbb E_\mu[e^{\zeta}] \end{array}$$

where $\widetilde P$ is a tilted law and $\zeta$ captures variance-type fluctuation.

3. Propagation-of-Chaos in Fisher Information: Lemma 3.4 yields

$$\mathrm{FI}(\mu^{[k]}\Vert\pi^{\otimes k}) = O\left(\frac{d\,k^3}{N^2}\right)$$

under stationarity.

4. Control of Tilted-KL Term: Applying a second LSI-to-Rényi type argument:

$$\mathrm{KL}(\widetilde P\Vert\mu) \lesssim (q-1)C_{\rm LSI} \mathsf{RF\!I}_q \le \tfrac{1}{2} \mathsf{RF\!I}_q + O\left(\frac{d\,k^3 q^2}{N^2}\right),$$

absorbing half of the $\mathsf{RF\!I}_q$ term to the left.

5. Exponential Moment Bound: Via concentration of Lipschitz functionals under the stationary law and LSI,

$$\log \mathbb E_{\mu^{[k]}}[e^\zeta] = \widetilde O\left(\frac{d\,k^3 q^2}{N^2}\right)$$

with a hierarchical coupling controlling the relevant Lipschitz constant as $O(k/N)$.

Aggregating these, one concludes the sharp

$$\mathsf R_q = \widetilde O\left(\frac{d\,k^3 q^2}{N^2}\right)$$

for general $k$-marginals, so for $k=1$,

$$\mathsf R_q(\mu^1\Vert\pi) = O\left(\frac{d q^2}{N^2}\right).$$

5. Dependence on System Parameters and Sharpness

The established bound reveals several sharp regimes:

• Scaling Laws: The bound scales as $q^2/N^2$ up to polylog factors. In the Gaussian case, the $1/N^2$ and $d$ dependences are sharp, with an explicit expansion yielding $\mathsf R_q = \Theta(dq/N^2)$.
• Optimality: The only apparent suboptimality is the $q^2$ dependence; the true scaling is conjectured to be linear in $q$.
• Reduction to KL: In the limit $q\downarrow 1$, the bound recovers the $O(1/N^2)$ KL divergence propagation-of-chaos law.
• Concentration Transfer: The corollary

$\mu^{[k]}(A) \le 2\, [\pi^{\otimes k}(A)]^{1/2}$

for $N \gtrsim \sqrt{d}\,k^{3/2}$ demonstrates high-probability transfer of concentration phenomena.

The stationary setting is essential for (i) closed-form score functions, (ii) uniform application of LSI to all conditional marginals, and (iii) sub-Gaussian concentration under the stationary law. Extensions to time-dependent (non-stationary) dynamics remain open.

6. Illustrative Gaussian Example

Consider $V(x) = \frac{1}{2}\|x\|^2$, $W(x) = \frac{\lambda}{2}\|x\|^2$. The stationary law $\mu^{1:N}$ is Gaussian with explicit block-covariance. For $N \gg k$,

$$\mathsf R_q\left(\mu^{1:k}\Vert\pi^{\otimes k}\right) = \frac{d\,q\,\lambda^2}{4(1+\lambda)^2}\, \frac{k[k(1+\lambda)^2-(2\lambda+1)]}{N^2} + O(d N^{-3}).$$

For $k=1$, this results in $\mathsf R_q(\mu^1\Vert\pi) = \Theta(d q / N^2)$, certifying the optimality of the $1/N^2$ and $d$ dependence, with the only discrepancy in the $q$ exponent relative to the general bound.

7. Broader Context and Implications

Propagation of chaos in Rényi divergence generalizes classical mean-field limit results beyond KL or Wasserstein metrics, providing a fine-grained, risk-sensitive quantification of independence in high-dimensional particle systems. The entropic and functional-analytic approach employed relies crucially on log-Sobolev inequalities, Donsker–Varadhan large deviations, and hierarchical couplings, synthesizing ideas from kinetic theory, concentration of measure, and information theory. These results further clarify the rates and mechanisms underlying empirical measure convergence in statistical physics and related domains (Zhang, 15 Jan 2026). High-dimensional sharpness and explicit scaling laws lay the foundation for subsequent investigations into temporal dynamics and non-stationary extensions.