From Statics to Dynamics: Physics-Aware Image Editing with Latent Transition Priors

Abstract: Instruction-based image editing has achieved remarkable success in semantic alignment, yet state-of-the-art models frequently fail to render physically plausible results when editing involves complex causal dynamics, such as refraction or material deformation. We attribute this limitation to the dominant paradigm that treats editing as a discrete mapping between image pairs, which provides only boundary conditions and leaves transition dynamics underspecified. To address this, we reformulate physics-aware editing as predictive physical state transitions and introduce PhysicTran38K, a large-scale video-based dataset comprising 38K transition trajectories across five physical domains, constructed via a two-stage filtering and constraint-aware annotation pipeline. Building on this supervision, we propose PhysicEdit, an end-to-end framework equipped with a textual-visual dual-thinking mechanism. It combines a frozen Qwen2.5-VL for physically grounded reasoning with learnable transition queries that provide timestep-adaptive visual guidance to a diffusion backbone. Experiments show that PhysicEdit improves over Qwen-Image-Edit by 5.9% in physical realism and 10.1% in knowledge-grounded editing, setting a new state-of-the-art for open-source methods, while remaining competitive with leading proprietary models.

• The paper introduces a transition-centric editing model that uses latent transition priors to predict continuous physical state changes for greater realism.
• It employs a dual-stream architecture combining explicit textual-visual physical reasoning with implicit latent queries to align diffusion-guided image edits with causality.
• Empirical results show significant improvements on benchmarks, outperforming baselines in physical realism and robustness across diverse dynamic scenarios.

Physics-Aware Image Editing via Latent Transition Priors: Summary and Analysis

Motivation and Problem Formulation

Prevailing instruction-based image editing models, including state-of-the-art instruction-following diffusion and unified multi-modal models, operate primarily as static image-to-image mapping systems. These systems demonstrate high semantic alignment but routinely violate physical plausibility, particularly for instructions that require causal reasoning governed by physical laws (e.g., simulating refraction, deformation, or phase transitions). Such limitations become more acute as user demands shift from basic object modifications to manipulations contingent on material properties or interactions.

This paper reframes the problem: rather than treating editing as a direct mapping from $(I_\text{src}, T_\text{edit}) \mapsto I_\text{tgt}$, it introduces a transition-centric paradigm where editing is modeled as a predictive physical state transition—a continuous temporal evolution governed by explicit causal mechanisms. This stands in contrast to black-box pixel updates, addressing the underspecification of transition dynamics present in legacy paired-image datasets.

PhysicTran38K: Video-Based Supervision for Physical State Transitions

To supervise models for the transition-based formulation, the authors construct PhysicTran38K, a high-fidelity video dataset with 38,620 curated video-instruction pairs. The dataset is systematically annotated with hierarchical physics categories spanning five primary physical domains (Mechanical, Biological, Thermal, Optical, Material), 16 sub-domains, and 46 discrete transition types (e.g., light refraction, melting, seed germination). The data synthesis pipeline involves:

• Structured Generation: Video synthesis follows templates encoding start state, trigger, transition process, and final state, generated using large-scale video diffusion models with static-camera constraints.
• Principle-Driven Verification: Video plausibility is automatically adjudicated by GPT-5-mini, which produces transition-specific physical principles. Videos are strictly filtered for physical law compliance, rejecting those with insufficient principle alignment.
• Constraint-Aware Annotation: Final editing instructions are paired with transition reasoning and logical constraints, explicitly excluding samples that violate physical laws or exhibit annotation-verification discrepancy.

This meticulous curation is strictly designed to foster principle acquisition over mere instruction mimicry, as demonstrated by negligible gains realized when similar models are trained on instruction-only datasets.

PhysicEdit: Dual-Stream Physics-Aware Editing Framework

The proposed PhysicEdit architecture builds on Qwen-Image-Edit but introduces a textual-visual dual-thinking mechanism:

• Physically-Grounded Reasoning Branch: A frozen Qwen2.5-VL-7B MLLM generates structured, explicit physical reasoning in text that details the constraints and expected causal mechanisms for each edit.
• Implicit Visual Thinking Branch: Inspired by MetaQuery, $K$ learnable transition queries are appended to the input. During training, these queries are supervised using features extracted from intermediate video keyframes (using DINOv2 for structure and the model VAE for texture), and learn to implicitly encode the delta in latent space representing the transition.
• Timestep-Aware Feature Modulation: To synchronize guidance with the diffusion process, the network dynamically interpolates between structural (DINOv2) and textural (VAE) features as a function of denoising step, facilitating coherent macro-to-micro physical state formation.

All parameters except the MLLM and backbone are optimized end-to-end, with loss terms targeting both diffusion output and query alignment to temporally-distributed video frame representations.

Empirical Evaluation

Quantitative Results

PhysicEdit achieves substantial and consistent gains over both open-source and proprietary baselines on multiple benchmarks:

Model	PICABench Score	KRISBench Score
Qwen-Image-Edit	61.26	65.56
ChronoEdit	58.67	70.96
PhysicEdit	64.86	72.16

• PhysicEdit outperforms Qwen-Image-Edit by 5.9% (PICABench, physical realism) and 10.1% (KRISBench, knowledge-grounded editing).
• Gains concentrate in categories requiring dynamic causal simulation and adherence to physical principles: light source effects (+14.97), deformation (+12.10), causality (+10.28), and refraction (+6.96 over baseline).
• On RISE-Bench, which directly probes temporal/causal editing, PhysicEdit more than doubles the base model’s overall score.

Ablation and Architectural Analysis

• Textual-Visual Duality: Isolated physically-grounded reasoning boosts logical/structural dimensions (Mechanics), while implicit visual thinking uplifts optics-centric effects. Their combined effect is optimal; neither is redundant.
• Transition Feature Fusion: Continuous timestep-aware weighting between DINOv2 and VAE is superior to hard switching, enhancing overall coherence and physical correctness.
• Dataset Design: Training on instruction-mimicry datasets (e.g., PICA-100K) yields marginal improvement, whereas the principle-driven PhysicTran38K enables significant generalization, particularly on benchmarks requiring out-of-distribution physical reasoning.
• Implicit vs Explicit Reasoning: Compared to explicit frame synthesis (ChronoEdit), PhysicEdit's latent transition queries provide better performance and robustness, avoiding error accumulation and high compute cost associated with pixel-space rollouts.

Qualitative Results

PhysicEdit demonstrates high fidelity for challenging instructions, manifesting correct optical effects (refraction, illumination decay), physically valid deformations, and globally coherent scene transitions. Competing models recurrently hallucinate, produce geometry/color mismatches, or break causal integrity.

Implications and Future Directions

This work offers a robust and scalable approach to bridging semantic editing and physical realism, leveraging video-based transition priors and hybrid reasoning architectures. The implications are significant for simulation-driven content creation, virtual world modeling, and educational/dynamics-oriented visual generation. However, increased capacity for physically plausible manipulation amplifies risks associated with photorealistic forgeries, emphasizing the need for parallel research on synthetic detection.

Potential future avenues include:

• More granular explicit modeling of physical laws or symbolic priors.
• Extension to multimodal or multi-view video editing and non-vision physics simulation.
• Tight integration with 3D-aware and physics engine-based neural architectures.
• Research into adversarial robustness for regularizing “plausibility” under adversarial, open-world triggers.

Conclusion

The transition from static, instruction-based editing towards physics-governed dynamic generation, as embodied by PhysicEdit and PhysicTran38K, marks a methodological inflection point for physically-correct visual AI. Explicitly modeling latent transition dynamics—not simply input-output pairs—is a prerequisite for deep causal reasoning in generative models. This work delivers a high-performing, open-source physics-aware editor and a generalizable framework for dynamic scene manipulation, substantiated by rigorous evaluation and ablation. The approach lays critical groundwork for future AI systems that operate safely and reliably in environments where physical logic is paramount.