Flow Matching for Diffusion Training

• The paper introduces Flow Matching, an ODE-based method that regresses a neural velocity field to transform noise into data without simulation.
• It utilizes analytic interpolation paths, such as linear and trigonometric curves, to achieve direct likelihood estimation and streamline training.
• Extensions like Local Flow Matching and Contrastive objectives enhance stability, reduce memory use, and deliver state-of-the-art performance across various domains.

Flow Matching is a simulation-free, ODE-based training and sampling framework for generative modeling, offering a stable and efficient alternative to classical diffusion probabilistic models. Flow Matching (FM) directly regresses a neural velocity field that transports noise samples to data samples along analytically constructed interpolation paths. This paradigm generalizes the probability-flow ODE formulation, enables direct likelihood estimation, supports various optimal-transport and diffusion-inspired trajectories, and yields state-of-the-art performance in image, tabular, and sequential domains. Recent advances include Local Flow Matching (LFM), contrastive objectives, explicit marginal losses, parameter-efficient alignment with diffusion models, and extensions to reinforcement learning, policy learning, speech enhancement, and self-supervised representation learning.

1. Mathematical Foundation and ODE Formulation

Flow Matching builds upon continuous-time neural ODEs. The generative transformation is expressed as the solution map of:

$$\frac{dx(t)}{dt} = v(x(t), t; \theta), \quad x(0) \sim p_0$$

where $v: \mathbb{R}^d \times [0, T] \to \mathbb{R}^d$ is a neural network velocity field, and $p_0$ is the base density (often Gaussian noise or smoothed data) (Xu et al., 2024). Under suitable regularity (Lipschitz continuity), this yields a diffeomorphic and invertible mapping from noise to data, or vice versa.

The velocity field $v$ is trained so that the ODE's induced continuity equation transports the input distribution to the target. In contrast to denoising score matching, which regresses $\nabla_x \log p_t(x)$ under a forward SDE, FM targets the deterministic ODE drift underlying diffusion, bypassing stochastic gradient estimation and variance weighting hassles (Lipman et al., 2022, Holderrieth et al., 2 Jun 2025).

2. Flow Matching Objectives and Loss Functions

For a specified interpolation path $\phi(t)$ between $x_l\sim p_0$ and $x_r\sim p_1$, typical choices are: linear OT path $I_t = x_l + t(x_r - x_l)$, or trigonometric interpolation $$I_t = \cos(\tfrac{1}{2}\pi t)x_l + \sin(\tfrac{1}{2}\pi)x_r$$. The ground truth target velocity is the analytic derivative $d\phi(t)/dt$.

The canonical FM loss is:

$$L(\theta) = \mathbb{E}_{t, x_l, x_r} \| v(\phi(t), t; \theta) - \frac{d\phi(t)}{dt} \|^2$$

For Gaussian diffusion or OT paths, $d\phi/dt$ admits closed form, enabling $L^2$ regression without SDE simulation or score estimation (Lipman et al., 2022, Xu et al., 2024). FM is compatible with CNFs, providing exact and unbiased log-likelihoods via the instantaneous change-of-variables formula.

Explicit Flow Matching (ExFM) further refines this by integrating out path endpoint variability, yielding conditional averaged targets and provably reduced estimator variance (Ryzhakov et al., 2024).

3. Local, Progressive, and Contrastive Extensions

Local Flow Matching (LFM): LFM decomposes a single large FM problem into $N$ incremental blocks, each matching a small diffusion step from $p_{n-1}$ to $p^*_n = \mathrm{OU}_0^{\gamma_n}(p_{n-1})$. Each block trains a compact velocity network over its interval, matching analytic OT or trigonometric paths (Xu et al., 2024). This architecture yields faster convergence and reduced memory, with generation guarantees:

$\chi^2(p_N \| q) = O(\epsilon^{1/2})$

where $\epsilon$ bounds FM error per block.

Progressive Reflow: Progressive Reflow curricula the straightening by initially dividing the time interval into local windows, applying FM piecewise, and merging adjacent windows in stages, decreasing optimization difficulty and improving stability. Aligned $v$-prediction focuses the loss on velocity direction rather than magnitude, reducing sample error in high-energy domains (Ke et al., 5 Mar 2025).

Contrastive Flow Matching: In conditional FM (e.g., class, text), flow uniqueness is violated, leading to mode collapse. Contrastive FM introduces a negative-pair loss penalizing similarity of flows between differing conditions:

$$L_{\Delta FM}(\theta) = L_{FM}^{(cond)}(\theta) - \lambda L_{contrast}(\theta)$$

This encourages disjoint latent flows, sharper conditional separation, and accelerates convergence, with empirically validated reductions in FID and denoising steps (Stoica et al., 5 Jun 2025).

4. Training and Sampling Algorithms

FM training simply samples endpoint pairs, interpolates at a random $t$, computes the analytic velocity target, and regresses via Adam:

• Draw $x_l \sim p_0$, $x_r \sim p_1$, $t \sim U[0,1]$
• $x_t = \phi(t)$
• Target $u^* = d\phi/dt$
• Optimize $v(\phi(t), t; \theta) \approx u^*$

For LFM, blocks are trained independently; see:

for n in range(N): # Sample data for block n x_l ~ p_{n-1}, x_r ~ p_{n}^* for t in [0, 1]: phi_t = I_t(x_l, x_r) loss = ||v_n(phi_t, t; θ_n) - dphi/dt||^2 update θ_n via Adam

Sampling proceeds by integrating the learned ODE(s) backward from noise to data, using Dormand–Prince or RK4 solvers. LFM achieves generation in $N$ sequential ODE solves, each with reduced memory/compute (Xu et al., 2024).

5. Theoretical Guarantees and Comparative Analysis

Flow Matching admits direct contraction results in $\chi^2$-divergence (and hence KL, TV) under bounded FM error and invertibility assumptions. For incremental LFM steps of size $\gamma$:

$$\chi^2(p_n \| q) \leq e^{-2\gamma n} \chi^2(p_0 \| q) + C\epsilon^{1/2}/(1-e^{-2\gamma})$$

Reverse flows generate $q_0$ guaranteeing $\chi^2(p\|q_0) \leq C\epsilon^{1/2}$, and thus $KL = O(\epsilon^{1/2})$, $TV = O(\epsilon^{1/4})$ (Xu et al., 2024). ExFM is mathematically equivalent to CFM in gradient but achieves faster, lower variance convergence (Ryzhakov et al., 2024). FM defined via optimal transport aligns with the dynamic OT solution for large data and moderate shifts, but its interpolation coefficients degrade under finite sample regimes; diffusion bridges become preferable for severe distribution discrepancies and scarce data (Zhu et al., 29 Sep 2025).

6. Empirical Performance and Applications

FM and its variants have demonstrated competitive or state-of-the-art results across domains:

Method	Dataset	FID (↓)	NLL	Remarks
LFM (Xu et al., 2024)	CIFAR-10	8.45		5× fewer batches than InterFlow
LFM (Xu et al., 2024)	ImageNet-32	7.00		3× fewer batches than baseline
LFM (Xu et al., 2024)	Tabular (MINIBOONE)		9.95	Best among methods
LFM (Xu et al., 2024)	Flowers	71.0		After 4-step distillation
FM w/ OT (Lipman et al., 2022)	CIFAR-10	6.35	2.99	Best BPD and FID
CFM (Schusterbauer et al., 2023)	FacesHQ SR	1.36		SOTA SR/PSNR, SSIM
SFMSE (Zhou et al., 25 Sep 2025)	Speech			RTF=0.013, 1-step, matches 60-step diffusion
Streaming FM (Jiang et al., 28 May 2025)	RoboMimic			95–100% imitation, 3.5–4.5ms latency
StraightFM (Xing et al., 2023)	CIFAR-10/Latent	2.82/8.86		One-step or few-step SOTA

FM is integral in high-resolution latent upsampling (CFM), reinforcement learning via ODE-to-SDE conversion (Flow-GRPO), imitation learning (Streaming Flow Policy), speech enhancement (SFMSE), and joint SSL generative/representation learning (FlowFM) (Schusterbauer et al., 2023, Liu et al., 8 May 2025, Ukita et al., 17 Dec 2025).

7. Implementation Choices and Practical Details

FM and its extensions use standard deep architectures: fully connected MLPs for tabular/2D, UNets for image/latent inputs (with channel multipliers [1,2,...]), and Transformers with ViT-style patches for sensor/time series. Adam optimizer with β₁=0.9, β₂=0.999, learning rate $1e^{-4}$–$5e^{-4}$, exponential decay. ODE solvers include RK4, Dormand–Prince. Divergence estimation for log-likelihood is via Hutchinson's trick or analytic Jacobian where feasible (Xu et al., 2024).

Block time steps ($\gamma_n$) may use geometric schedules, tuning $(c, \rho)$ for optimal convergence. Interpolation (OT/trigonometric) adapted to task. For non-density data, initial OU diffusion $\delta \approx 0.1$ regularizes support for theoretical guarantees. In policy/reinforcement domains, streaming actions in action space yields lowered latency and tight sensorimotor integration (Jiang et al., 28 May 2025).

• Local Flow Matching Generative Models (Xu et al., 2024)
• Contrastive Flow Matching (Stoica et al., 5 Jun 2025)
• Flow Matching for Generative Modeling (Lipman et al., 2022)
• ProReflow: Progressive Reflow with Decomposed Velocity (Ke et al., 5 Mar 2025)
• Explicit Flow Matching: On The Theory of Flow Matching Algorithms with Applications (Ryzhakov et al., 2024)
• High-Performance SSL by Joint Training of Flow Matching (Ukita et al., 17 Dec 2025)
• Diffusion Bridge or Flow Matching? A Unifying Framework (Zhu et al., 29 Sep 2025)
• Exploring Straighter Trajectories of Flow Matching with Diffusion Guidance (Xing et al., 2023)
• Flow Diverse and Efficient: Learning Momentum Flow Matching (Ma et al., 10 Jun 2025)
• Streaming Flow Policy (Jiang et al., 28 May 2025)
• Shortcut Flow Matching for Speech Enhancement (Zhou et al., 25 Sep 2025)
• Boosting Latent Diffusion with Flow Matching (Schusterbauer et al., 2023)
• Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (Schusterbauer et al., 2 Jun 2025)
• An Introduction to Flow Matching and Diffusion Models (Holderrieth et al., 2 Jun 2025)
• Flow-GRPO: Training Flow Matching via Online RL (Liu et al., 8 May 2025)
• Unraveling the Connections between Flow Matching and Diffusion Probabilistic Models (Song et al., 2024)

Flow Matching constitutes a robust generative modeling paradigm that unifies ODE-based transport, optimal transport interpolants, and modern deep learning for efficient high-quality synthesis, conditional generation, and beyond.