WSA$_1$: a 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

Published 4 Jul 2026 in cs.RO | (2607.03941v1)

Abstract: Recent advances in embodied AI have established robot foundation models (RFMs) as the dominant approach for generalist robotic systems to date. By leveraging imitation learning on extensive robot demonstrations, RFMs have achieved impressive capabilities in mapping visual observations and language instructions to continuous robotic actions. However, current RFMs lack an inherent ability to reason about physical dynamics and the causal effects of robot behaviors on the 3D physical world. This creates a fundamental mismatch between 2D-centric visual perception and 3D-centric embodied interaction, severely limiting the generalization ability of RFMs in real-world tasks.To address this gap, we present WSA$_1$, a novel RFM built upon proposed 3D-Centric World-Spatial-Action modeling paradigm. It not only learns 3D world-aware visual thought for future robot behaviors, but also models mutual constraints between 3D world state transitions and robotic actions to enhance behavior generalization. Notably, WSA$_1$ achieves highly data-efficient pre-training with 6k hours of expert demonstration data (only 1k hours from real robot), while delivering competitive manipulation performance (93% success rate) on RoboTwin2.0 simulation benchmark and achieving +20% average boosted performance over state-of-the-art RFMs on real-world robot control tasks. These results reveal that generalizable RFM can be attained without large-scale real robot data when paired with 3D-centric world-action joint modeling, which offers a practical and affordable pathway to generalist robotic systems.

Abstract PDF Upgrade to Chat

Authors (11)

Summary

The paper presents a novel 3D-centric world-spatial-action model that unifies 3D dynamics, 2D visual cognition, and action synthesis for robust robotic control.
It employs a Mixture-of-Transformers with bidirectional causal attention to seamlessly integrate world changes with robot actions, enhancing generalization.
Empirical results demonstrate over 20% performance improvement on diverse real-world tasks, highlighting superior data efficiency and control robustness.

WSA₁: A 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

Introduction and Motivation

WSA $_1$ introduces a novel 3D-centric World-Spatial-Action (WSA) modeling paradigm for robot foundation models (RFMs), directly addressing the canonical limitation of existing Vision-Language-Action (VLA) and World-Action Model (WAM) approaches: insufficient modeling of 3D dynamics and world–action interdependencies. While VLAs and WAMs exhibit strong performance in mapping visual/language inputs to physical actions, they largely remain 2D-centric and lack explicit mechanisms for capturing how continuous robot actions interact with, and causally affect, the 3D physical world. This deficiency fundamentally constrains their generalization in unconstrained and diverse real-world tasks.

WSA $_1$ advances RFM design by explicitly co-modeling 3D world dynamics, 2D visual cognition, and 3D-conditioned action generation in a unified shared latent space, via a Mixture-of-Transformers (MoT) architecture. The work asserts that explicit 3D physical modeling is essential for robust manipulation policies, and empirically demonstrates substantial gains in both data efficiency and control robustness, even when pre-trained with only 1K hours of real-robot data, outperforming existing open-source RL and RFM solutions by a +20% margin across real-world tasks.

Prevailing RFM Paradigms: Positioning WSA

WSA $_1$ is grounded in a critical analysis of prior RFM paradigms, which can be categorized as follows:

VLA Models: Unidirectional, 2D-centric mappings from multi-modal visual semantics (RGB frames, language) to continuous action spaces, often failing to encode the causal and geometric constraints of 3D scene evolution.
2D-Centric WAMs: Joint 2D video prediction and action modeling, offering some temporal consistency but still lacking explicit 3D physical reasoning.
3D-Centric WAMs: Extend predictive modeling to 3D geometry, but maintain a "predict-then-execute" unidirectional approach, without closing the loop between world changes and action inference.
WSA (Proposed): Fully joint, bidirectional modeling of 3D world transitions, spatial visual predictions, and action generation, capturing mutual constraints and dynamics.
Figure 1: Comparative overview of modeling paradigms, with WSA explicitly coupling 3D world dynamics, visual reasoning, and actions in a unified latent space.

The hierarchical and bidirectional nature of the WSA approach directly targets the generalization bottleneck of prior models by enforcing the mutual dependencies between world changes and the agent's physical actions.

Model Architecture and Learning Objectives

WSA $_1$ utilizes a Mixture-of-Transformers backbone structured into three domain-specific experts:

2D Spatial Expert (2D-SE): Predicts 3D-consistent visual trajectories (i.e., subgoal visual tokens) guiding future robot behaviors.
3D Spatial Expert (3D-SE): Models and predicts transformations in the 3D scene caused by action sequences, leveraging pre-trained 3D scene encoders.
3D Action Expert (3D-AE): Learns 3D inverse dynamics via diffusion models, mapping intended future world states to continuous robot action trajectories.

A central innovation is a bidirectional causal attention mechanism, enforcing cross-expert dependencies for both forward (action→scene) and inverse (scene→action) reasoning. This shared latent design unifies predictive modeling, planning, and control in a single architecture.

Figure 2: Schematic of WSA $_1$ ’s architecture, highlighting the three Mixture-of-Transformer experts and the bidirectional causal attention pathways.

The three learning objectives correspond to:

Action-conditioned 3D world modeling (forward dynamics)
3D-consistent visual thinking (goal-conditioned visual generation)
3D inverse dynamics (scene-grounded action synthesis)

WSA $_1$ is instantiated in both base (WSA $_1$ -B, 3B params) and large (WSA $_1$ -L, 6B params) configurations, both leveraging high-capacity vision-language encoders (Qwen3-VL, Wan2.2) and pre-trained 3D scene modules (e.g., Depth Anything).

Data Efficiency via Multi-Source Pre-Training

Anticipating the prohibitive cost of large-scale real-robot data, WSA $_1$ leverages a heterogeneous data mixture of only 6K hours, including:

Synthetic simulation (e.g., RoboTwin2.0, InternData-A1)
Egocentric human manipulation videos (EgoDex)
Real-robot demonstrations (RoboChallenge, AgiBot-World)

This strategy, enabled by robust 3D-centric modeling and multi-task learning, sharply limits data requirements without sacrificing generalization, as demonstrated in extensive benchmarks.

Empirical Evaluation: Numerical Results

Real-World Robot Tasks: On seven diverse real-world tabletop manipulation tasks with both uni- and bi-manual robots, WSA $_1$ -B delivers a success rate of 77.5% (completeness: 82.7%), while WSA $_1$ 0-L achieves 80.3%/86.5%. Compared to the top baseline ( $_1$ 1), this is a >+20% improvement in success rate across tasks.

RoboTwin2.0 (Simulation): On this hard-mode, dual-arm simulation benchmark, WSA $_1$ 2-L achieves a 93.1% mean success rate, exceeding all other open-source models (best baseline: Fast-WAM, 91.8%), and matching closed-source MotuBrain (96.1%) within 3 p.p., despite much lower data usage. The improvement is especially pronounced on spatially complex, long-horizon, and contact-rich tasks.

LIBERO: WSA $_1$ 3-L attains a 98.2% average success rate, competitive with or superior to state-of-the-art models in spatial, goal-conditioned, and object-centric settings.

Figure 3: Comparative evaluation of WSA variants and ablations on the RoboTwin2.0 benchmark.

Figure 4: Ablation results showing the effect of world-spatial-action pre-training and joint modeling on task success rate.

Ablative and Qualitative Analysis

Ablation studies reveal that all three modeling capacities—visual thinking, world prediction, and action generation—are necessary for optimal performance. Importantly, bidirectional causal modeling in a shared space outperforms individual or sequential training by at least 13 p.p. on challenging benchmarks.

Joint WSA pre-training and fine-tuning confer additional gains of 12–13 p.p. over action-only or sequential paradigms.

Figure 5: Visualization of WSA $_1$ 4’s multimodal predictions: observed scene, visual planning (subgoal images), 3D world prediction, and actual executed trajectory.

Qualitative visualizations show robust spatio-temporal alignment between model intent (visual, 3D, and action predictions) and realized manipulation behaviors, supporting the claim that WSA $_1$ 5 substantially closes the simulation-to-real-world policy gap.

Theoretical and Practical Implications

WSA $_1$ 6 demonstrates that 3D-centric mutual modeling of physical world transitions and actions is critical for attaining generalizable, robust, and data-efficient robotic agents. The paradigm naturally enables transfer across distinct embodiments and environments with minimal additional real-robot supervision, unlike prior models locked into high-cost data scaling. The mutual constraints enforced via bidirectional attention induce strong physical priors and facilitate transfer of physical commonsense.

Practically, the WSA approach is significant for scalable robotics deployment, reducing the resource barrier for learning generalist policies and enabling rapid adaptation to novel tasks and platforms.

Theoretically, this work points to latent space architectures in embodied AI that go beyond tokenization or static mapping, capturing causal and geometric dependencies critical for embodied intelligence. The conceptual and technical advances suggest a promising avenue for the development of autonomous systems capable of hybrid perceptual and physical reasoning, bridging the gap to more human-level embodied cognition.

Conclusion

WSA $_1$ 7 advances the field of embodied AI by establishing joint 3D-centric world, spatial, and action modeling as the foundation for robust, transferable, and data-efficient robotic control. The integration of bidirectional world–action constraints, unified in a Mixture-of-Transformers backbone, produces strong gains in generalization and real-world robustness. Empirical results on simulation and real-robot benchmarks support the core thesis: explicit 3D world-spatial-action modeling is necessary and sufficient for high-performance generalist robot foundation models using affordable data scales, and sets the stage for further exploration in latent space causality, transfer learning, and cross-embodiment generalization in embodied agents.