RoboBrain 2.5: Depth in Sight, Time in Mind

Abstract: We introduce RoboBrain 2.5, a next-generation embodied AI foundation model that advances general perception, spatial reasoning, and temporal modeling through extensive training on high-quality spatiotemporal supervision. Building upon its predecessor, RoboBrain 2.5 introduces two major capability upgrades. Specifically, it unlocks Precise 3D Spatial Reasoning by shifting from 2D pixel-relative grounding to depth-aware coordinate prediction and absolute metric constraint comprehension, generating complete 3D manipulation traces as ordered keypoint sequences under physical constraints. Complementing this spatial precision, the model establishes Dense Temporal Value Estimation that provides dense, step-aware progress prediction and execution state understanding across varying viewpoints, producing stable feedback signals for downstream learning. Together, these upgrades extend the framework toward more physically grounded and execution-aware embodied intelligence for complex, fine-grained manipulation. The code and checkpoints are available at project website: https://superrobobrain.github.io

• The paper introduces a dual-path architecture that integrates precise 3D spatial reasoning with step-wise temporal progress estimation.
• It uses a standardized (u, v, d) representation with curriculum training to generate accurate 3D trajectories for physically grounded manipulation.
• The model’s robust bidirectional temporal value estimation provides reliable closed-loop feedback, improving error recovery in real-world tasks.

RoboBrain 2.5: Precise 3D Spatial Reasoning and Dense Temporal Progress Modeling in Embodied AI

Motivation and Limitations of Prior Art

Vision-language-action (VLA) models have advanced the integration of perception, reasoning, and high-level planning for embodied agents, but existing architectures fall short in physical reliability and execution-level grounding. Most prior models are inherently limited in two aspects: (1) metric blindness due to 2D pixel-based spatial grounding that neglects depth and absolute scale, resulting in inadequate metric-level control and failure under physical constraints, and (2) open-loop temporal modeling, lacking intrinsic mechanisms for dense, step-wise progress estimation, resulting in poor execution monitoring, limited adaptive recovery, and the inability to provide continuous feedback required for robust real-world manipulation. RoboBrain 2.5 directly tackles these issues through architectural, data, and supervision innovations that span both spatial and temporal axes.

Figure 1: New features of RoboBrain 2.5: depth-aware 3D spatial reasoning and dense step-aware temporal value estimation, leading to robust manipulation and closed-loop progress feedback.

Architectural Innovations

RoboBrain 2.5 is built upon the Qwen3-VL backbone with a dual-path enhancement strategy: (1) Precise 3D Spatial Reasoning and (2) Dense Temporal Value Estimation. The integration of both pathways enables RoboBrain 2.5 to bridge the gap between high-level plan generation and physically grounded execution.

Precise 3D Spatial Reasoning

The model formalizes spatial action prediction as ordered sequences of keypoints $\left\{(u_t, v_t, d_t)\right\}_{t=1}^T$, leveraging monocular RGB input augmented by camera intrinsics. This $(u, v, d)$ representation admits trivial geometric projection, leading to accurate 3D trajectory generation. The 3D spatial interface is curriculum-trained to handle:

• 3D Spatial Referring: Precise localization of objects via hierarchical and metric-aware queries.
• 3D Spatial Measuring: Metric-scale estimation (e.g., distances, clearances).
• 3D Spatial Tracing: Generation of collision-free, ordered 3D manipulation traces, explicitly satisfying start/end/trajectory metric constraints.

This approach standardizes multimodal supervision over hybrid 2D and 3D grounded datasets, supporting extensive multi-task and cross-modal training.

Figure 2: Training data distribution, showing the scale and domain composition for spatial, temporal, and general data, with dominance of dense value estimation and metric 3D data.

Dense Temporal Value Estimation

The model generalizes task progress estimation as a hop-wise, normalized temporal value modeling problem. Each task trajectory is segmented into fine-grained frames; progress is recursively predicted as a normalized change relative to remaining or completed distance, always constrained to $[0,1]$ via hop-based update rules. This formulation is inherently robust to sampling rate and out-of-distribution drift, enabling closed-loop reward modeling and error recovery.

Multi-perspective fusion (incremental, forward-anchored, backward-anchored) ensures stability over diverse temporal directions and mitigates error accumulation. This dense value function serves as a high-fidelity reward signal for RL, rather than relying on sparse success-only supervision.

Training Corpus and Curriculum

RoboBrain 2.5 is trained on a systematically mixed corpus exceeding 12 million multimodal samples. The data is explicitly balanced between:

• General MLLM data (semantic and open-domain reasoning)
• High-resolution spatial grounding and point tracing data (from LVIS, Pixmo-Points, RoboPoint, CA-1M, ScanNet, and manipulation video sources)
• Hierarchically labeled temporal datasets (EgoDex, AGIBot, DROID, LIBERO, RoboCasa) with expert keyframe segmentation and hop-based progress annotation

This mixture is essential for retaining strong general visual-linguistic priors while enabling metric-precise trace generation and dense, embodiment-invariant value estimation. Anti-forgetting strategies are employed to prevent overfitting during metric-specific fine-tuning.

Infrastructure and Training Strategy

A notable aspect of RoboBrain 2.5 is the use of a hybrid hardware platform and a deeply optimized training stack, including cross-accelerator support (NVIDIA and MTT). The system employs uneven pipeline parallelism to balance ViT and transformer stages, and a unified dynamic memory pre-allocation mechanism for multi-modal long-sequence workloads.

Training follows a two-stage scheme:

1. Foundational Spatiotemporal Learning: Multi-task, multi-modal learning of general visual perception, 2D grounding, and initial temporal ordering.
2. Specific Spatiotemporal Enhancement: Fine-tuning on 4.1M samples for 3D quantitative reasoning and dense value estimation, mixing in randomly sampled prior data for stabilization.

Empirical Results

2D and 3D Spatial Reasoning

RoboBrain 2.5 achieves substantial improvements over both general MLLM and embodied model baselines on comprehensive benchmarks for 2D spatial reasoning (CV-Bench, CrossPoint, RoboSpatial, RefSpatial, EmbSpatial). Notably, it surpasses Gemini-3-Pro-Preview, Qwen3-VL, and Mimo-Embodied in both accuracy and generalization. In 3D, RoboBrain 2.5 is the first model to demonstrate high accuracy in metric 3D start/end and full-trajectory prediction (TraceSpatial-Bench), outperforming all baselines by significant margins.

Figure 3: RoboBrain 2.5's predicted traces compared with ground-truth on TraceSpatial-Bench; consistent spatial compliance from start to end.

Temporal Progress Prediction

RoboBrain 2.5 delivers bidirectionally consistent and highly robust progress estimation across real, simulated, and human egocentric video domains, as measured by forward and reverse rank correlation (VOC$^+$/VOC$^-$). Competing VLMs (including GPT-5.2 and Gemini) achieve strong forward VOC but exhibit drastic drops under time-reversal, reflecting a lack of genuine temporal understanding.

Figure 5: Frame-wise hop changes and accumulated progress curves predicted by RoboBrain 2.5 on diverse manipulation tasks.

Figure 7: Reconstructed progress is robust to input stride, demonstrating invariance to sampling interval.

Bidirectional consistency strategies reduce reward hacking during RL deployment, and artificial disturbances in the real system (e.g., target movement) are correctly reflected in progress drops, supporting policy adaptation and recovery.

Figure 9: Robustness to real-world perturbation—policy recovers from misalignment guided by dense progress estimation signals.

Implications and Future Directions

RoboBrain 2.5 demonstrates that (1) depth-aware 3D tracing can be achieved in VLA models using standardized $(u,v,d)$ supervision and cross-modal data aggregation, and (2) dense, self-consistent temporal value estimation enables robust closed-loop reward modeling decoupled from external expert labels. The combination of these techniques enables reliable deployment in long-horizon, contact-rich tasks, and provides a scalable blueprint for future embodied AI systems.

Theoretical implications include the practical bounding of progress estimation using hop-wise supervision, facilitating convergence and OOD robustness in reinforcement learning. On the hardware side, cross-accelerator compatibility provides a scalable template for distributed multi-modal model training.

Prospective developments include unification of spatiotemporal understanding with generative prediction (world models), testing and adaptation across diverse robot embodiments (including mobile and humanoid platforms), scalable model variants for edge deployment, and the creation of a self-evolving data engine powered by dense value verification.

Conclusion

RoboBrain 2.5 represents a comprehensive evolution of embodied AI, integrating depth-aware 3D spatial reasoning with dense, drift-resistant temporal value estimation. Extensive empirical evaluation confirms strong, hardware-invariant state-of-the-art results on rigorous 2D/3D spatial reasoning and temporal progress benchmarks. The model's ability to generalize across hardware, tasks, and domains, while delivering actionable, physically grounded outputs, positions it as a reference point for subsequent advances in generalist embodied intelligence. The dense reward models demonstrated here are likely to become foundational components in future closed-loop learning and deployment regimes for real-world robotics.