An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces

Abstract: Let $q(x)$ and $p(x)$ denote density functions on the $n$-dimensional Euclidean space, and let $p_i(\cdot|y_1,..., y_{i-1},y_{i+1},..., y_n)$ and $Q_i(\cdot|x_1,..., x_{i-1},x_{i+1},..., x_n)$ denote their local specifications. For a class of density functions $q$ we prove an inequality between the relative entropy $D(p||q)$ and a weighted sum of the conditional relative entropies $D(p_i(\cdot|Y_1,..., Y_{i-1},Y_{i+1},..., Y_n) ||Q_i(\cdot|Y_1,..., Y_{i-1},Y_{i+1},..., Y_n))$ that holds for any $p$. The weights are proportional to the logarithmic Sobolev constants of the local specifications of $q$. Thereby we derive a logarithmic Sobolev inequality for a weighted Gibbs sampler governed by the local specifications of $q$. Moreover, this inequality implies a classical logarithmic Sobolev inequality for $q$, as defined for Gaussian distribution by L. Gross. This strengthens a result by F. Otto and M. Reznikoff. The proof is based on ideas developed by F. Otto and C. Villani in their paper on the connection between Talagrand's transportation-cost inequality and logarithmic Sobolev inequality.

• The paper introduces an inequality connecting global relative entropy with a sum of conditional relative entropies weighted by local logarithmic Sobolev constants.
• It leverages Fokker-Planck dynamics and coupling arguments to generalize criteria for LSIs in weakly dependent, non-product measures.
• The work provides sharp thresholds for LSIs in structured systems and connects these results to quadratic transportation-cost inequalities in high dimensions.

Relative Entropy Inequalities and Logarithmic Sobolev Inequalities in Euclidean Spaces

Introduction

This paper establishes an inequality connecting global relative entropy and sums of conditional relative entropies, weighted by local logarithmic Sobolev constants, for a class of probability densities $q(x)$ on $\mathbb{R}^n$. The main advance is a new approach to deducing logarithmic Sobolev inequalities (LSIs) for product measures (and, by extension, Gibbs measures with weak dependencies) in Euclidean space under less restrictive assumptions. The framework leverages gradient structures, Fokker-Planck dynamics, and recent connections between entropy, transportation inequalities, and curvature conditions. The treatment generalizes and partially strengthens existing criteria by Otto, Reznikoff, and others, particularly regarding the relaxation of the symmetry and boundedness requirements for mixed partials of $V(x) = -\log q(x)$.

Main Results

The central inequality established relates the global relative entropy $D(p \| q)$ for an arbitrary density $p$ to a weighted sum of conditional (local specification) relative entropies: $$D(p \| q) \leq \frac{1}{\rho} \sum_{k=1}^n p_k D(p^{(k)}(\cdot|Y^{(k)}) \| Q^{(k)}(\cdot|Y^{(k)})),$$ where $p_k$ is the logarithmic Sobolev constant associated with the $k$-th local specification $Q^{(k)}$, and $\rho$ is a constant specified by an explicit, but subtle, condition involving the mixed partial Hessians of $V(x)$. The formulation allows for general non-product $q$, subject to natural curvature and coupling control on the local specifications, and quantifies the deficit in LSI in terms of the off-diagonal block structure of the Hessian.

Theorem 2 asserts that, under three technical assumptions—local LSI, block lower bounds on the Hessian, and controlled cross-block interactions—the global measure $q$ satisfies a logarithmic Sobolev inequality with constant $\rho$. Furthermore, this result implies, via the Otto-Villani argument, that $q$ also satisfies a quadratic transportation-cost inequality with the same constant.

Theorem 1, which is instrumental for the proof of Theorem 2, gives a sharp upper bound for the global entropy in terms of the conditional entropies, establishing a form of tensorization for weakly dependent coordinates. Notably, the key coupling assumption (Assumption 3) is strictly weaker than the spectral norm constraint used by Otto-Reznikoff, which is demonstrated to be critical in certain high-dimensional Toeplitz/nearest neighbor dependence scenarios.

Key Techniques and Proof Innovations

The proof strategy innovatively exploits evolution under the Fokker-Planck equation (i.e., the so-called entropy dissipation method). Interpolation between $p$ and the reference measure $q$ is carried out via the diffusion semigroup generated by $\Delta - \nabla V \cdot \nabla$, with the dynamics analyzed by differentiating entropy along the flow. This setup enables mapping the global entropy decay problem to a sum of local entropy decays plus a carefully controlled error term arising from the interaction structure.

Key technical components include:

• The introduction of a matrix $A(x, \xi)$ capturing the effect of cross-block Hessian (second partial) entries, whose norm bounds measure the deviation from the case of independent coordinates.
• The use of an auxiliary contractivity argument for a weighted Gibbs sampler (randomly updating local blocks), demonstrating strict decay of relative entropy per iteration.
• A refined approximation lemma ensuring smoothness and compact support for densities, which is crucial for analytical control of semigroups and derivative estimates.

In handling non-constant interaction terms (i.e., non-Gaussian $q$), the analysis carefully separates diagonal block and off-diagonal block contributions, and recasts off-diagonal effects as a cost in the LSI constant.

The paper also provides explicit comparison with Otto-Reznikoff [10], both in the form of the coupling criterion and in the setting where the Hessian is constant (Gaussian case). It is shown that for certain structured dependence (e.g., Toeplitz block matrices from spin systems or Markov chains), the new criterion gives a sharp threshold for validity of LSI, while the prior result is suboptimal.

Numerical Significance and Contradictory Cases

The work provides constructed Toeplitz matrix examples where the previously established Otto-Reznikoff spectral condition fails, but the present, weaker block-norm condition holds, thus establishing that the result is genuinely stronger for structured interaction matrices. In these cases, LSIs with optimal constants are derived for measures $q = \exp(-V)$ with off-diagonal interactions, which cannot be handled optimally by prior tensorization approaches.

The explicit connection to transportation inequalities is also quantitatively significant. By deducing a Wasserstein-2 transportation-cost inequality with the same constant as the LSI, measure concentration properties for weakly dependent random fields in Euclidean space immediately follow. This links geometric and functional-inequality phenomena in high dimensions, as anticipated by Otto-Villani.

Implications and Future Directions

The main theoretical implication is a clarified, more widely applicable criterion for the validity of LSIs for measures on Euclidean space with product-like but weakly coupled potentials. This has direct consequences for the analysis of convergence and concentration in high-dimensional Gibbs measures, Markov random fields, and spin systems with non-trivial interaction structure.

Practically, the results provide tools for bounding mixing times of associated Markov semigroups (Ornstein-Uhlenbeck, Langevin, Gibbs samplers) in non-product, weakly dependent settings. Since entropy dissipation (LSI) controls both exponential convergence rates and measure concentration, these results can be leveraged in statistical physics, high-dimensional probability, and sampling-based algorithms in statistics and machine learning.

Further, by making precise the interaction between local tensorization (via conditional LSI) and global geometric structure (via off-diagonal Hessian terms), the paper suggests a unified strategy for treating LSIs in both discrete (e.g., spin systems) and continuous (e.g., Euclidean Gibbs fields) domains.

Possible future directions include:

• Extension to infinite-dimensional settings or to manifolds with (possibly non-Euclidean) metrics.
• Application to non-Euclidean, possibly singular or discrete, product spaces.
• Quantitative improvement and computationally tractable verification of the block-norm criterion, especially in large systems with structured dependencies.

Conclusion

This work advances the theoretical understanding of logarithmic Sobolev inequalities and their connection to relative entropy in high-dimensional Euclidean spaces with weakly dependent coordinates. The introduced criterion allows for broader classes of non-product measures to admit sharp LSI and associated concentration inequalities, with direct ramifications for convergence analyses in statistical physics and high-dimensional probability. The techniques also deepen the interplay between geometric, analytic, and probabilistic perspectives in the theory of functional inequalities.

For the full exposition, proofs, and further implications, see "An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces" (1206.4868).