Hamiltonian Generative Flows

• Hamiltonian Generative Flows are deep generative models that use Hamiltonian dynamics to construct invertible and volume-preserving transformations with exact likelihood evaluation.
• They leverage symplectic integrators and learned Hamiltonians to ensure stable, reversible dynamics and efficient sampling in high-dimensional spaces.
• HGFs unify paradigms like normalizing flows, diffusion models, and flow matching while offering theoretical guarantees through information geometry and universal approximation.

Hamiltonian Generative Flows (HGFs) are a class of deep generative models that construct invertible and volume-preserving transformations by leveraging the principles of Hamiltonian dynamics. They generalize and unify a spectrum of continuous-time generative modeling paradigms—including normalizing flows, diffusion models, and flow matching—while providing increased expressivity, theoretical guarantees, and rigorous connections to physical and information-geometric structures. HGFs enable efficient density modeling and sampling by encoding probability distributions as time-evolved states under parameterized Hamiltonian systems, ensuring both exact invertibility and tractable likelihood evaluation.

1. Mathematical Foundations

HGFs operate on phase space variables $z = (q, p) \in \mathbb{R}^{2d}$ (position and momentum). The core generative mechanism is based on time evolution governed by a (learned) Hamiltonian $H_\theta(q, p)$ via Hamilton’s equations: $$\begin{aligned} \dot{q} &= \frac{\partial H_\theta}{\partial p}, \ \dot{p} &= -\frac{\partial H_\theta}{\partial q}. \end{aligned}$$ This gives a smooth, invertible, and volume-preserving map $\Phi_T: z_0 \mapsto z_T$ after integrating from $t=0$ to $t=T$.

The symplectic structure is encoded via the canonical two-form

$\omega = \sum_{i=1}^d dq_i \wedge dp_i.$

Liouville’s theorem ensures that $\Phi_T$ exactly preserves phase-space volume for all $t$, i.e.,

$$\left| \det D\Phi_T(z) \right| = 1 \quad \forall z,$$

where $D\Phi_T(z)$ is the Jacobian of the flow (Aich et al., 28 May 2025, Rezende et al., 2019, Toth et al., 2019).

HGFs use (neural) parameterizations of $H_\theta$ (often as summations $K_\theta(p) + U_\theta(q)$ with small MLPs) and numerically integrate Hamilton’s equations with symplectic integrators such as the leapfrog (Störmer-Verlet) scheme, which ensures both invertibility and volume preservation at finite step size.

2. Invertibility, Volume Preservation, and Exact Likelihood

The volume-preserving property allows HGFs to compute exact likelihoods without the cost of Jacobian determinant evaluation necessary in conventional flows: $$\log p(x) = \log\, \mathbb{E}_{z_0 \sim \mathcal{N}(0, I)} \left[ p_\theta(x | \Phi_T(z_0)) \right]$$ when using a Gaussian latent prior. For flows where only positions are observed, a variational posterior over momenta is introduced and the evidence lower bound (ELBO) is maximized accordingly (Aich et al., 28 May 2025, Toth et al., 2019, Rezende et al., 2019). These guarantees hold for discrete-time implementations via symplectic integrators at arbitrary step size, eliminating the need for computationally intensive determinant or trace calculations (Toth et al., 2019, Aich et al., 28 May 2025).

3. Network Architectures and Model Variants

Several architectures instantiate HGFs:

• Symplectic Generative Networks (SGNs): Realize learned Hamiltonians $H_\theta(q, p)$ with fully connected layers under spectral normalization. The generator comprises an invertible encoder, Hamiltonian time evolution, and an invertible decoder. Leapfrog integration is used for stable, reversible dynamics (Aich et al., 28 May 2025).
• Hamiltonian Normalizing Flows (NHF/PDE-NHF): Adopt a separable Hamiltonian $H=K(p)+V(q)$. $K(p)$ is typically quadratic, and $V(q)$ is parameterized by Deep Sets or other permutation and translation invariant neural architectures to enforce physical symmetries. These models are applied to both density estimation and the solution of kinetic PDEs, and enable rapid sampling by composing learned invertible maps (Souveton et al., 7 May 2025, Toth et al., 2019).
• Generalized Hamiltonian Generative Flows (PH-ODE and Oscillation HGF): Relax the assumption $F_\theta = \nabla U$ by using learned, possibly non-conservative force fields $F_\theta(x, t)$, allowing flows to encompass and strictly generalize diffusion and flow-matching models. For example, setting $F=0$ recovers denoising score-matching, and harmonic force fields yield Oscillation HGFs (Holderrieth et al., 2024).

A summary of key model differences:

Model	Hamiltonian Param.	Conservativity	Volume Preservation	Sampling
SGN	Neural $H_\theta(q,p)$	Yes	Exact	Fast + stable
NHF/PDE-NHF	$K$ fixed, $V$ neural	Yes	Exact	Fast
PH-ODE/Oscillation	$F_\theta$ learned	Optional	Exact	General
Equivariant HGF	$H_\theta$ with group invariance	Yes	Exact	Invariant under $G$

4. Theoretical Guarantees: Universal Approximation and Information Geometry

Hamiltonian flows are universal approximators of volume-preserving diffeomorphisms on compact sets. Specifically, any $C^1$ volume-preserving map isotopic to the identity can be approximated uniformly by time-$T$ flows generated by a neural Hamiltonian integrated via leapfrog, with quantifiable error bounds: $$\sup_{z \in \Omega} \| \Phi_T(z) - \Phi(z) \| \leq C_1 (nL)^{-1/(2d)} + C_2\, \Delta t^2$$ where $n$ is width, $L$ depth, and $\Delta t$ the leapfrog step size (Aich et al., 28 May 2025).

Information-theoretically, any volume-preserving invertible map $f$ preserves entropy and mutual information: $H(Z) = H(X)$ and $I(X;Z) = H(X)$, in contrast with stochastic mappings (e.g., VAEs) where $I(X;Z) < H(X)$ (Aich et al., 28 May 2025).

Geometrically, exponential families equipped with the Fisher–Rao metric and canonical symplectic structure allow the flow to respect information geometry, further motivating the Hamiltonian framework for invertible mappings (Aich et al., 28 May 2025).

5. Extension to Symmetries and Equivariance

HGFs support the learning of distributions equivariant or invariant to Lie group actions. Enforcing invariance of the Hamiltonian under group generators (vanishing Poisson bracket $\{g_k, H\}=0$) ensures the learned flow commutes with symmetry actions. Practically, invariance can be implemented via Lagrange penalty terms or group-invariant network design (e.g., parameter tying, invariant neural networks) (Rezende et al., 2019). This equips HGFs with improved data efficiency, generalization, and supports learning of disentangled subspaces when group factorizations are present.

6. Algorithmic Implementation, Stability, and Computational Complexity

Symplectic integrators—especially the leapfrog scheme—are central for discretizing flow maps. Each step is exactly invertible (by reversing sign of step size), ensures phase-space volume preservation, and allows for adaptive integration via step-size control to enforce global error bounds (Aich et al., 28 May 2025, Souveton et al., 7 May 2025). Backward error analysis shows energy drift is $O(\Delta t^2)$ over exponentially long times. Stability criteria depend on spectral norms of $H_\theta$'s Hessian, e.g., $\Delta t < 2/\sqrt{L_H}$ for leapfrog case.

Complexity analysis reveals that SGNs/HGFs avoid per-step determinant computation, with total cost $O(Td)$ for $T$ steps/dimension $d$, and memory $O(T+d)$. In contrast, standard normalizing flows scale as $O(K C_J(d))$ in determinant cost and $O(Kd)$ memory for $K$ layers, with $C_J(d)$ the cost of a $d \times d$ determinant (Aich et al., 28 May 2025, Toth et al., 2019).

7. Connections to Quantum Simulation and Theoretical Generality

There exists an isomorphism between the continuity equation for classical flows and a time-dependent Schrödinger equation under a derived "continuity Hamiltonian". Specifically, the transformation $\Psi_t(x) = \sqrt{p_t(x)}$ enables the use of a quantum computer to efficiently prepare "qsamples"—coherent encodings of modeled densities—by simulating the evolution under the Hamiltonian constructed from the learned vector field $v_t(x)$. Quantum sampling allows for mean estimation and property testing with algorithmic advantages over classical methods for heavy-tailed observables (Layden et al., 9 Oct 2025).

8. Empirical Performance and Applications

Empirical investigations across density estimation, physics-inspired systems, and high-dimensional image synthesis confirm the expressivity and efficiency of HGFs and their variants. Hamiltonian flows match or exceed baseline performance on multimodal density benchmarks with fewer steps and substantially lower computational cost. Oscillation HGFs achieve competitive FID scores on CIFAR-10 and FFHQ datasets with fewer function evaluations compared to diffusion models (Holderrieth et al., 2024). Applications span generative modeling, physical simulation (e.g., Vlasov-Poisson kinetic equations), and quantum information tasks (Souveton et al., 7 May 2025, Layden et al., 9 Oct 2025).

9. Limitations and Future Directions

HGFs require accurate estimation of gradients (and, where required, Hessians) of neural Hamiltonians. Extending to very high-dimensional settings and non-connected/discrete symmetry groups remains an open challenge. Scalability to large datasets and integration with adaptive neural architectures are active areas of research. Empirical validation on complex, real-world data remains ongoing (Aich et al., 28 May 2025, Rezende et al., 2019).

---

References:

• "Symplectic Generative Networks (SGNs): A Hamiltonian Framework for Invertible Deep Generative Modeling" (Aich et al., 28 May 2025)
• "Hamiltonian Normalizing Flows as kinetic PDE solvers: application to the 1D Vlasov-Poisson Equations" (Souveton et al., 7 May 2025)
• "Hamiltonian Generative Networks" (Toth et al., 2019)
• "Equivariant Hamiltonian Flows" (Rezende et al., 2019)
• "Hamiltonian Score Matching and Generative Flows" (Holderrieth et al., 2024)
• "Wavefunction Flows: Efficient Quantum Simulation of Continuous Flow Models" (Layden et al., 9 Oct 2025)