Coupled Diffusion Sampling for Training-Free Multi-View Image Editing

Abstract: We present an inference-time diffusion sampling method to perform multi-view consistent image editing using pre-trained 2D image editing models. These models can independently produce high-quality edits for each image in a set of multi-view images of a 3D scene or object, but they do not maintain consistency across views. Existing approaches typically address this by optimizing over explicit 3D representations, but they suffer from a lengthy optimization process and instability under sparse view settings. We propose an implicit 3D regularization approach by constraining the generated 2D image sequences to adhere to a pre-trained multi-view image distribution. This is achieved through coupled diffusion sampling, a simple diffusion sampling technique that concurrently samples two trajectories from both a multi-view image distribution and a 2D edited image distribution, using a coupling term to enforce the multi-view consistency among the generated images. We validate the effectiveness and generality of this framework on three distinct multi-view image editing tasks, demonstrating its applicability across various model architectures and highlighting its potential as a general solution for multi-view consistent editing.

• The paper introduces a coupled diffusion sampling algorithm that aligns pre-trained 2D and multi-view models to produce view-consistent edits.
• It leverages implicit 3D regularization via a coupling energy term, yielding improvements in metrics like PSNR, SSIM, and multi-view consistency.
• The method is training-free and efficient, generalizing across various backbones while reducing artifacts without explicit 3D optimization.

Coupled Diffusion Sampling for Training-Free Multi-View Image Editing

Introduction

This work introduces a training-free, inference-time method for multi-view consistent image editing by coupling pre-trained 2D image editing diffusion models with multi-view diffusion models. The approach addresses the challenge of achieving view-consistent edits across multiple perspectives of a 3D scene or object, a task where existing methods either require explicit 3D representations (e.g., NeRF, 3D Gaussian Splatting) or retraining of multi-view models for each editing task, both of which are computationally expensive and data-intensive. The proposed method leverages implicit 3D regularization by enforcing that the generated 2D image sequences adhere to a pre-trained multi-view image distribution, achieved through a novel coupled diffusion sampling algorithm.

Figure 1: Applications of coupled diffusion sampling, enabling off-the-shelf 2D editing models to produce view-consistent multi-view edits for spatial editing, stylization, and text-based relighting.

Methodology

Coupled Diffusion Sampling

The core contribution is a coupled sampling algorithm for diffusion models. Given two diffusion models—$\epsilon_{\theta^{A}}$ (e.g., a 2D editing model) and $\epsilon_{\theta^{B}}$ (e.g., a multi-view model)—the goal is to generate samples $x^A$ and $x^B$ that are both faithful to their respective data distributions while being spatially and semantically aligned. This is formalized by introducing a coupling energy term $U(x, x') = -\frac{\lambda}{2}\|x - x'\|_2^2$, which penalizes discrepancies between the two samples.

The coupled sampling modifies the standard DDPM update by adding the gradient of the coupling term to the score function at each denoising step:

$$\nabla_x \mathcal{J}^{i}(x, x') = \nabla_x\log p^{i}(x) + \nabla_x U(x, x'), \quad i\in \{A, B\}$$

This results in a soft regularization that encourages the two sample trajectories to remain close throughout the diffusion process, while each remains within its own model's prior.

Figure 2: Overview of coupled sampling. (a) Standard DDPM sampling produces independent, misaligned samples. (b) Coupled sampling introduces a coupling term, pulling the two sample paths together for spatial and semantic alignment.

The algorithm is efficient, requiring only a feed-forward pass through both models at each step, with minimal additional computation for the coupling term. The method is agnostic to the underlying diffusion model architecture and can be applied in both pixel and latent spaces.

Implementation Details

• Model Alignment: The noise schedules of the 2D and multi-view models are aligned to ensure effective coupling.
• Sampling: 50–100 denoising steps are used, with the coupling term scaled by the noise level at each step.
• Resource Requirements: The method requires GPU memory sufficient for both models, but can be optimized by sequential model loading.
• Generalization: The approach is demonstrated with various backbones (e.g., Stable Diffusion 2.1, SDXL, MVDream, MV-Adapter) and can be extended to flow-based models (e.g., Flux) by transforming their velocity fields to score functions.

Experimental Results

Multi-View Spatial Editing

The method is evaluated on spatial editing tasks using Magic Fixup as the 2D editing model. The coupled approach achieves higher PSNR, SSIM, and multi-view consistency (MEt3r) compared to baselines, including per-image editing, image-to-multiview conditioning, and compositional diffusion methods. Qualitatively, the method preserves object identity and produces consistent edits across views, avoiding flickering and geometric artifacts.

Figure 3: Qualitative comparison on multi-view spatial editing. Baselines exhibit identity loss and flickering, while coupled sampling achieves both editing targets and multi-view consistency.

Multi-View Stylization

For stylization, ControlNet is used as the 2D model. The coupled method outperforms mesh-based and compositional baselines in temporal and subject consistency, as measured by VBench and MEt3r. The method maintains high CLIP scores for prompt fidelity and is strongly preferred in user studies.

Figure 4: Multi-view stylization results. Coupled sampling yields consistent stylization across views, outperforming prior compositional and mesh-based methods.

Multi-View Relighting

The method is applied to both environment map-based (Neural-Gaffer) and text-based (IC-Light) relighting. Coupled sampling eliminates flickering artifacts and achieves competitive reconstruction and consistency metrics, even when compared to NeRF-based approaches.

Figure 5: Qualitative comparison on environment map-based relighting. Coupled sampling avoids flickering and view-dependent artifacts present in other methods.

Figure 6: Text-based relighting with IC-Light and coupled sampling, producing diverse and consistent multi-view relighting results.

Generalization and Analysis

The coupling strategy is validated across different multi-view backbones (MVDream, MV-Adapter) and with flow-based models (Flux). The method consistently improves realism and spatial alignment, even when the underlying multi-view models exhibit synthetic or CGI-like artifacts.

Figure 7: Coupling in different multi-view models. Coupled sampling improves realism and reduces synthetic appearance in multi-view outputs.

Figure 8: Image space coupling with Flux. Coupled samples are spatially aligned while remaining faithful to their respective prompts.

Effect of Coupling Strength

Ablation studies show that increasing the coupling strength $\lambda$ improves reconstruction up to a point, after which over-regularization degrades multi-view consistency and can cause sample collapse.

Figure 9: Effects of coupling strength. Increasing $\lambda$ improves alignment but excessive values degrade consistency.

Stochasticity in Sampling

The effectiveness of the coupling term depends on the use of stochastic samplers (e.g., DDPM). Deterministic samplers (e.g., ODE-based) tend to produce simple averages, lacking the corrective behavior enabled by stochastic noise injection.

Figure 10: Sampler comparison. Stochastic samplers allow natural guidance, while deterministic samplers yield simple averages.

Limitations

• Computational Overhead: Running both models in parallel increases memory and compute requirements.
• Residual Inconsistency: While improved, perfect 3D consistency is not guaranteed; further regularization (e.g., fitting a NeRF post-hoc) can reduce residual artifacts.
• Scalability: The method is currently limited by the capacity of the underlying multi-view model and the number of views it can process.

Implications and Future Directions

The proposed coupled diffusion sampling framework provides a general, training-free solution for multi-view consistent image editing, leveraging existing 2D and multi-view diffusion models. This paradigm can be extended to other domains, such as video editing (by coupling with video diffusion models), or to other modalities where cross-model consistency is desired. The method's flexibility and efficiency make it suitable for real-world applications where rapid, consistent multi-view edits are required without retraining or explicit 3D optimization.

Potential future developments include adaptive or selective coupling strategies to reduce computational overhead, integration with explicit 3D representations for further consistency, and application to domains beyond vision, such as audio-visual or multi-modal generation.

Conclusion

This work presents a principled, efficient approach for multi-view consistent image editing by coupling pre-trained 2D and multi-view diffusion models at inference time. The method achieves strong quantitative and qualitative results across spatial editing, stylization, and relighting tasks, generalizes across model architectures, and is robust to various sampling strategies. While some limitations remain, the approach represents a significant step toward practical, scalable, and consistent multi-view editing without the need for explicit 3D optimization or retraining.