Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model

Abstract: Recent foundation image and video generation models offer strong generalization and controllability, but their direct application to embodied scenarios is limited by requirements for multi-view consistency, geometric coherence, and robot embodiment constraints. Existing methods typically adapt foundation models with limited robot data, often sacrificing visual knowledge acquired during large-scale pre-training. We present Xiaomi-Robotics-U0, a 38-billion-parameter multimodal autoregressive model for unified embodied synthesis. It treats embodied generation as an extension of foundation image and video generation and jointly optimizes text-to-image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation. This unified framework preserves the generalization of the pre-trained world foundation model while adapting it to embodied settings. Xiaomi-Robotics-U0 is the first model to support high-quality multi-view scene generation across multiple robot embodiments and to introduce structured, controllable embodied transfer for fine-grained editing while preserving multi-view consistency and interaction dynamics. It achieves state-of-the-art results on single-step and sequential generation tasks, outperforming GPT-Image-2.0 in human evaluations of embodied scene generation and transfer, ranking first on World Arena for embodied video generation, and improving the out-of-distribution success rate of pi_0.5 from 36.9% to 63.2% on challenging real-world manipulation tasks. These results show that foundation world models can serve both as embodied world models and scalable data engines for embodied intelligence. Code and checkpoints are available at https://robotics.xiaomi.com/xiaomi-robotics-u0.html.

• The paper introduces a unified multimodal autoregressive model for embodied synthesis that directly integrates visual, textual, and control tokens to meet stringent robotic constraints.
• It leverages a 38B-parameter architecture co-trained on diverse datasets, achieving superior multi-view consistency, semantic grounding, and robust performance benchmarks.
• The model enhances real-world robotic policy training through effective data augmentation and sequential world modeling, significantly improving out-of-distribution success rates.

Unified Embodied Synthesis with the Xiaomi-Robotics-U0 World Foundation Model

Introduction and Motivation

Xiaomi-Robotics-U0 proposes a unified, large-scale multimodal autoregressive architecture for embodied synthesis, bridging the gap between foundation visual generation models and the stringent requirements of embodied AI. The model addresses foundational issues in applying open-domain generative models to robotics: multi-view consistency, geometric and kinematic constraints, and downstream policy compatibility. Unlike conventional adaptation strategies that fine-tune foundation models on relatively limited robot data and risk severe overfitting or catastrophic forgetting, Xiaomi-Robotics-U0 treats embodied synthesis as a direct extension of world foundation modeling, co-training on general and domain-specific robotic datasets in a single unified objective.

The model is designed to support five core tasks: text-to-image generation, image editing, embodied scene generation, embodied transfer (i.e., systematic scene alteration under geometric and control constraints), and embodied video generation (i.e., sequential rollout of robot-environment interactions). These capabilities are not treated in isolation but as tractable modalities within a shared token space under an autoregressive next-token prediction objective.

Figure 1: Embodied and general capabilities of Xiaomi-Robotics-U0, illustrating initial observations, transfer samples, and video keyframes across tasks and modalities.

Figure 2: Task inputs and outputs overview, showing the diversity of conditions handled in Xiaomi-Robotics-U0.

Model Architecture and Training Paradigm

Xiaomi-Robotics-U0 leverages a 38B-parameter architecture, initialized from EMU3.5 and built atop Qwen-3-32B, incorporating an IBQ tokenizer for efficient image quantization. The model extends the vocabulary to include image, text, and robot control tokens, thereby collapsing all input/output modalities into a homogeneous token stream.

The architecture eschews task-specific prediction heads, in contrast to many prior embodied world models, and instead configures all generative sequence tasks—image, scene, editing, transfer, video—as variants of next-token prediction. This allows for efficient parameter sharing and facilitates auto-labeling and multitask scaling.

Figure 3: High-level overview of the model structure, showing unified end-to-end training under next-token autoregression and the inference extension mechanism.

A FlashAR$^+$ vertical prediction head is introduced, performing anti-diagonal decoding of visual tokens for target regions, enabling parallelization within image regions and up to $\sim$83x inference speedup on $1024 \times 1024$ images compared to baseline AR decoding. vLLM integration further increases throughput in large-scale batched serving scenarios.

Dataset, Annotation, and Task Curation

The training corpus encompasses 9.5M single-step samples (56.4B tokens) and 2.6M multi-frame video clips (49.6B tokens), drawn from six domains:

• General image-text pairs (ShareGPT4V and related corpora)
• Real and simulated embodied manipulation (AgiBotWorld-Beta, Open X-Embodiment, MiBot, RoboTwin2.0, GenieSim, InternData)
• Autonomous driving scenarios (Cosmos-Drive-Dreams)
• Egocentric recordings (EgoWalk)
• 3D reconstruction data (ScanNet++, DL3DV-10K)
• Game and synthetic environments (Open-P2P)

To bridge annotation scarcity and enable fine-grained supervision, a Qwen3-VL-235B powered automatic labeling pipeline generates compositional scene descriptions, dimension-decomposed embodied annotations (workspace, task/irrelevant objects, lighting, background), and trajectory subtask decompositions. Video Depth Anything is leveraged for depth estimation, keyframe segmentation, and geometric conditioning.

Figure 4: Construction of the multi-task single-step generation dataset, integrating T2I/X2I, geometry, camera/time control, and structure/semantic guidance for co-training.

Figure 5: Unified data labeling pipeline: semantic scene annotation and subtask sequence decomposition, supporting consistent robotic dataset supervision.

A deliberate sample weighting strategy addresses domain imbalance, encouraging long-tail visual and skill diversity while eliminating redundancy and error-prone robot trajectories.

Embodied Scene Generation and Transfer

Quantitative and Qualitative Results

For embodied transfer—with the model conditioned on scene description and multi-view depth—the benchmark comprises 300 samples (split by 'easy' and 'hard' edits against original scenes); metrics include SI-RMSE, $\delta_1$, AbsRel (geometry adherence), Canny F1/Precision/Recall (structural fidelity), and segmentation mIoU/Recall (semantic grounding).

Xiaomi-Robotics-U0 substantially outperforms GPT-Image-2 in all metrics: for example, SI-RMSE falls from 0.4007 to 0.1407 (easy split), and segmentation mIoU rises from 0.4105 to 0.7621, demonstrating precise geometric adherence and improved semantic grounding. Multi-view consistency is preserved not only quantitatively but also in qualitative assessments: compared with GPT-Image-2, which generates plausible but mutually inconsistent views violating depth and spatial alignment, Xiaomi-Robotics-U0 maintains strict cross-view coherence.

Figure 6: Qualitative embodied transfer results: editing and depth conditioning yield physically plausible, multi-view consistent outputs.

Figure 7: Comparison showing GPT-Image-2's geometric errors versus Xiaomi-Robotics-U0's adherence to 3D and consistent object layout.

The structured Any-to-Image capability further enables zero-shot data augmentation for downstream policy learning by generating diverse, scene-consistent keyframes—enhancing data variance for real-world manipulation.

Figure 8: Left: original multi-view keyframes; right: randomized visual conditions after zero-shot augmentation by the model.

Real-World Policy Impact

A direct evaluation of the utility of Xiaomi-Robotics-U0-generated augmentations demonstrates substantial improvements in out-of-distribution generalization. Policies trained on style-augmented demonstrations (generated with Xiaomi-Robotics-U0) achieve a jump in $\pi_{0.5}$ out-of-distribution success rate from 36.9% to 63.2%. The benefit is especially pronounced under extensive background and lighting perturbations, with the augmented policies achieving higher progress on long-horizon manipulation tasks without sacrificing in-distribution performance.

Figure 9: Empirical task completion progress: Xiaomi-Robotics-U0-Augmented policies outperform non-augmented baselines, especially under visual adversity.

Sequential World Modeling and Embodied Video Generation

The model extends its capacity beyond static generation, supporting long-horizon sequential rollout with interleaved visual-linguistic subtask conditioning and variable FPS dense sampling. Evaluation on WorldArena—a comprehensive embodied video generation benchmark—shows Xiaomi-Robotics-U0 ranks first overall among 100+ models, with top scores in instruction following, 3D perspectivity, motion smoothness, and interaction quality.

Figure 10: WorldArena leaderboard: Xiaomi-Robotics-U0 (UNIS codename) leads across visual, semantic, physical, and motion dimensions.

Figure 11: Qualitative video generation: initial scene and textual instruction lead to plausible, physically consistent robot-environment interaction sequences.

Generalization and Retained Open-Domain Generation

Despite extensive adaptation for embodied contexts, Xiaomi-Robotics-U0 retains strong compositional T2I and image editing performance, comparing competitively with domain-leading open-domain models except for cases demanding highly precise symbolic binding or multiple constraints.

Implications and Future Directions

Xiaomi-Robotics-U0 demonstrates that a unified autoregressive foundation world model can serve as a scalable, high-fidelity embodied data engine for both open-ended generative tasks and policy training in robotics. In contrast to specialized adaptation strategies that decouple foundation visual understanding from embodied constraints, this unified approach enables efficient transfer of semantic knowledge while meeting geometric, physical, and temporal requirements unique to robotics.

Practical implications include improved out-of-distribution robustness for manipulation policies, efficient augmentation pipelines for real-world robotic applications, and the deployment of closed-loop data generation engines capable of infinite synthetic trajectory production—critical for data-hungry robot learning frameworks.

Theoretical implications pertain to the viability of large-scale pre-training with structured multitask objectives as a means to achieve geometry-aware, temporally coherent reasoning across modalities and embodiments.

Future developments could focus on end-to-end multi-view image editing dispensing with explicit geometric intermediates, jointly optimized long-horizon scene and video rollout for improved temporal coherence, and expanding context windows for extended embodied sequence modeling.

Conclusion

Xiaomi-Robotics-U0 sets a new standard for unified multimodal embodied synthesis, demonstrating strong performance across diverse embodied tasks, robust generalization beyond the training distribution, and efficiency in both training and inference. By unifying foundation world modeling and embodied generation under a shared large-scale autoregressive framework, it substantiates the hypothesis that world foundation models, when properly adapted and supervised, can underpin scalable, agentic, and physically grounded AI systems (2607.11643).