Kuramoto Orientation Diffusion Models

Abstract: Orientation-rich images, such as fingerprints and textures, often exhibit coherent angular directional patterns that are challenging to model using standard generative approaches based on isotropic Euclidean diffusion. Motivated by the role of phase synchronization in biological systems, we propose a score-based generative model built on periodic domains by leveraging stochastic Kuramoto dynamics in the diffusion process. In neural and physical systems, Kuramoto models capture synchronization phenomena across coupled oscillators -- a behavior that we re-purpose here as an inductive bias for structured image generation. In our framework, the forward process performs \textit{synchronization} among phase variables through globally or locally coupled oscillator interactions and attraction to a global reference phase, gradually collapsing the data into a low-entropy von Mises distribution. The reverse process then performs \textit{desynchronization}, generating diverse patterns by reversing the dynamics with a learned score function. This approach enables structured destruction during forward diffusion and a hierarchical generation process that progressively refines global coherence into fine-scale details. We implement wrapped Gaussian transition kernels and periodicity-aware networks to account for the circular geometry. Our method achieves competitive results on general image benchmarks and significantly improves generation quality on orientation-dense datasets like fingerprints and textures. Ultimately, this work demonstrates the promise of biologically inspired synchronization dynamics as structured priors in generative modeling.

• The paper introduces a novel nonlinear score-based generative framework using Kuramoto dynamics for structured image synthesis.
• It leverages phase mapping and non-isotropic noise to efficiently reconstruct orientation-rich data, achieving lower FID and CMMD scores.
• The approach employs a hierarchical coarse-to-fine generation process and extends to spherical and physical datasets for broad applicability.

Kuramoto Orientation Diffusion Models: Structured Generative Modeling via Stochastic Synchronization

Introduction and Motivation

Kuramoto Orientation Diffusion Models introduce a nonlinear score-based generative modeling framework that leverages stochastic Kuramoto dynamics for image synthesis, particularly targeting orientation-rich data such as fingerprints, textures, and directional fields. The central innovation is the mapping of pixel values onto periodic angular phase variables, enabling the use of synchronization phenomena—ubiquitous in biological and physical systems—as an inductive bias for structured image generation. Unlike conventional diffusion models operating in Euclidean space with isotropic noise, this approach employs non-isotropic, phase-coupled noise dynamics, facilitating a more structured destruction and reconstruction process.

Figure 1: Schematic of globally and locally coupled Kuramoto orientation diffusion models, illustrating SDEs and representative image samples. Pixels are mapped to angular phase variables, with global or local sinusoidal coupling driving synchronization and structured noise dynamics.

Model Formulation

Stochastic Kuramoto Dynamics

The forward process is governed by a stochastic Kuramoto SDE, where each pixel (oscillator) phase $\theta^i_t$ evolves according to:

$$\frac{d\theta^i_t}{dt} = \frac{1}{N}\sum_{j=1}^N K(t) \sin(\theta^j_t - \theta^i_t) + K_{\text{ref}}(t) \sin(\psi_{\text{ref}} - \theta^i_t) + \sqrt{2D_t}\xi^i$$

Here, $K(t)$ is the time-dependent coupling strength, $K_{\text{ref}}(t)$ couples oscillators to a global reference phase $\psi_{\text{ref}}$, and $D_t$ is the noise variance. The model supports both global coupling (all-to-all) and local coupling (neighborhood-based), with phase wrapping ensuring periodicity.

The forward process synchronizes phase variables, driving the system toward a low-entropy von Mises distribution. The reverse process, parameterized by a learned score function, desynchronizes phases to reconstruct diverse image patterns.

Hierarchical Generation

The reverse generative process follows a coarse-to-fine paradigm: global structure is established first due to initial synchronization, and fine-scale details are progressively introduced via anti-coupling dynamics. This explicit hierarchy is a direct consequence of the nonlinear, non-isotropic phase interactions.

Figure 2: Hierarchical generation on CIFAR-10, showing coarse-to-fine progression from von Mises initialization to detailed image synthesis over 100 diffusion steps.

Figure 3: Hierarchical generation on SOCOFing fingerprints and Brodatz textures, with large-scale orientation patterns established before fine texture details emerge (300 diffusion steps).

Score Estimation and Training

Due to the nonlinear drift, the marginal distribution $p(\theta_t)$ is intractable. The score function is learned via local score matching, exploiting the wrapped Gaussian transition kernel:

$$p(\theta_t|\theta_{t-1}) \approx \frac{1}{\sqrt{4\pi D_{t-1}}} \sum_{k=-K}^{K} \exp\left(-\frac{(\theta_t - \theta_{t-1} - f(\theta_{t-1}, t-1) + 2\pi k)^2}{4D_{t-1}}\right)$$

Sinusoidal embeddings $[\sin(\theta), \cos(\theta)]$ are used for periodicity-aware neural networks, with outputs projected back onto the angular domain. Training employs Monte Carlo sampling from the truncated wrapped Gaussian, with $K=3$ and $M=5$ providing a balance between stability and efficiency.

Experimental Results

Orientation-Dense Datasets

On SOCOFing fingerprints and Brodatz textures, both globally and locally coupled Kuramoto models achieve significantly lower FID and CMMD scores than standard score-based generative models (SGM), especially at low diffusion step counts. The locally coupled variant further improves fidelity by aligning with spatial correlations.

Figure 4: Samples generated on SOCOFing fingerprints and Brodatz textures under varying denoising steps, demonstrating sharp and coherent orientation patterns.

Ground Terrain and General Images

On the ground terrain dataset, Kuramoto models outperform SGM across all step counts, maintaining coherent directional structure and material appearance.

Figure 5: Samples generated on the ground terrain dataset, showing preservation of orientation structure and texture realism.

On CIFAR-10, Kuramoto models excel at low step counts, but SGM surpasses them at 1000 steps, indicating that the synchronization bias is most beneficial for orientation-rich data and efficient sampling, but may limit expressiveness for natural images lacking strong orientation priors.

Figure 6: CIFAR-10 samples under varying denoising steps, illustrating the diminishing advantage of structured synchronization at higher step counts.

Spherical and Physical Data

The model generalizes to periodic domains, achieving competitive NLL on Earth and climate science datasets defined on the 2D sphere, and improving spectral plausibility in Navier-Stokes fluid velocity field generation by coupling phase and amplitude appropriately.

Figure 7: Learned density plots for Earth and climate science datasets, capturing both concentrated and dispersed spatial event patterns.

Implementation Considerations

• Computational Cost: Training incurs $\mathcal{O}(T)$ cost per step due to explicit forward chain simulation, but this can be mitigated by precomputing and caching forward SDE pairs.
• Network Architecture: U-Net with self-attention, sinusoidal timestep conditioning, group normalization, and GELU activations is used for score prediction.
• Parameter Schedules: Linear schedules for noise variance and coupling strengths are tuned to maintain $K_{\text{ref}} > D_t > K$ for global coupling, and $K_{\text{ref}} = K$ for local coupling.
• Inference: Both SDE and ODE solvers are supported, with phase wrapping enforced at each step.

Ablation and Metric Analysis

Ablation removing Kuramoto coupling (i.e., reference-only drift) degrades performance, confirming the necessity of non-isotropic phase synchronization. CMMD, a CLIP-based nonparametric metric, corroborates FID improvements, with Kuramoto models outperforming SGM by a substantial margin even at low step counts.

Theoretical and Practical Implications

The explicit use of synchronization dynamics as a structured prior enables efficient modeling of orientation-dense data, bridging concepts from nonlinear dynamics, computational neuroscience, and generative modeling. The hierarchical, interpretable generation process aligns with spectral biases in diffusion models, but is made explicit via phase coupling. The approach is extensible to other periodic or angular domains, including spherical data and physical fields.

Limitations include potential expressiveness constraints for natural images and training efficiency bottlenecks, though these are addressable via caching and hybrid modeling strategies. The method is particularly suited for applications in biometric security, medical imaging, and scientific visualization, but care must be taken regarding the ethical use of synthetic data.

Conclusion

Kuramoto Orientation Diffusion Models present a principled framework for generative modeling on periodic domains, leveraging biologically inspired synchronization dynamics to induce structured priors. The approach yields superior sample fidelity and efficiency on orientation-rich datasets, competitive performance on general images, and extensibility to spherical and physical data. Future work may explore integration with neural spiking data and further optimization of training efficiency, advancing the intersection of nonlinear dynamics and generative modeling.