Being-H0.5: Unified Vision-Language-Action Model

• Being-H0.5 is a foundational vision-language-action model that leverages large-scale, human-centric demonstrations to enable cross-embodiment robotic learning with state-of-the-art performance.
• Its architecture integrates a Mixture-of-Transformers and Mixture-of-Flow design, unifying multimodal inputs and a universal action space to bridge diverse robotic platforms.
• Advanced robustness mechanisms, including manifold-preserving gating and universal async chunking, ensure reliable performance under sensory shift and latency variations.

Being-H0.5 denotes a foundational Vision-Language-Action (VLA) model specifically developed for human-centric, cross-embodiment robotic learning, oriented toward robust generalization across a diverse array of robotic platforms. The framework leverages a unified action space, large-scale multimodal pre-training, a scalable Mixture-of-Transformers (MoT) backbone with a Mixture-of-Flow (MoF) module, and explicit robustness mechanisms to achieve state-of-the-art results in both simulation and real-world settings. The core principle is to treat egocentric human demonstration trajectories as a universal prior—conceptualized as a “mother tongue” for physical manipulation—facilitating action transfer and policy alignment across morphologically heterogeneous robots through a physically interpretable shared representation (Luo et al., 19 Jan 2026).

1. UniHand-2.0 Dataset and Data Pipeline

The UniHand-2.0 dataset underpins Being-H0.5’s human-centric approach, comprising over 35,000 hours of multimodal data (≈400 million samples, ≈120 billion tokens):

• Human Demonstrations: 16,000 hours of egocentric video, with hand trajectories parameterized via MANO model coordinates, augmented by camera extrinsics and contact events. Sources include Ego4D, EPIC-KITCHENS, Egocentric-10K, and proprietary UniCraftor RGB-D datasets.
• Robot Manipulation: 14,000 hours across 30 distinct robotic embodiments, unifying varied control spaces such as end-effector pose deltas, joint configurations, and gripper states, standardized at 30 Hz.
• Language and VQA Corpora: 5,000 hours (≈50B text tokens, 45.7B visual tokens) aggregate spatial- and intent-grounded language data.
• Filtering and Annotation: Data curation includes four-stage filtering (motion quality, manipulation relevance, handedness debiasing, text diversification). Vision and action streams are augmented by LLM-driven semantic alignment and paraphrase generation (Luo et al., 19 Jan 2026).

This large-scale pipeline allows integration of high-quality human motor data into the pre-training of cross-embodiment robotic policies, supporting both dense motion understanding and text-conditioned manipulation.

2. Human-Centric Learning Paradigm

The central conceptual advance of Being-H0.5 is its human-first policy learning architecture:

• Physical Prior: Human hand movements provide dense, causally structured representations of intent, affordance, and contact physics unachievable with limited robot data alone.
• Semantic Serialization: All modalities (vision, text, state, action) are formatted into QA-style sequences:

$$\mathcal{S} = [\,\mathcal{S}_Q;\,\mathcal{S}_A\,], \quad x_k = (m_k, C_k),\; m_k\in\{\text{vision},\text{text},\text{state},\text{action}\}$$

• Task Structure: The model supports motion generation (action given context), motion description (language given action + context), and motion continuation (predicting future action tokens), always grounded in the shared serialization format.
• Unified Pre-training Objective: Sequence modeling is cast as joint text-action prediction, integrating masked motion token prediction and hybrid flow-matching losses (see Sec. 6) (Luo et al., 19 Jan 2026).

This paradigm enables robots to “bootstrap” complex skills from humans, facilitating transfer across mechanical and sensory gaps.

3. Unified Action Space and Semantic Slotting

Being-H0.5 formulates a global, physically interpretable action space, partitioned into $K$ semantic “slots”:

• Slot Assignment: Each robot embodiment $e$ is mapped via a sparse function $\Phi_e$ such that

$$\mathbf{s} = \Phi_e( \mathbf{s}^{(e)} ), \quad \mathbf{a} = \Phi_e( \mathbf{a}^{(e)} )$$

with unspecified slots zeroed.

• Slot Types: Semantic elements include:
  • $\Delta$EEF Cartesian pose ($\Delta x, \Delta y, \Delta z$)
  • SE(3) axis-angle rotations
  • Joint angles (radians)
  • Gripper/finger widths
  • Mobile base velocities/headings
• Embedding Standards: All values retain physical units (e.g., no artificial scale normalization), maintaining fidelity between human and robot domains.
• Adaptation: Fine-tuning on low-resource platforms leverages slotwise adapter banks, updating only embodiment-specific slots during adaptation (Luo et al., 19 Jan 2026).

The unified slotting formalism ensures that skill transfer is possible even for robots with fundamentally different actuation architectures.

4. Model Architecture: MoT–MoF Design

The Being-H0.5 backbone features a hybrid architecture:

• Mixture-of-Transformers (MoT): Initialized from InternVL-3.5, each transformer layer interleaves two “expert” attention pathways:
  • Multimodal Understanding Head: Processes high-level vision-language input to generate contextual embeddings.
  • Action Generation Head: Produces continuous action outputs conditioned on context.
• Mixture-of-Flow (MoF) Expert: Decouples invariant motor primitives from embodiment-specific idiosyncrasies:
  • Shared Primitive Blocks: Capture universal reach, grasp, and object-avoidance behaviors.
  • Specialized Flow Modules: $E$ parallel flows $f_i$ are mixed via context-dependent Top-K soft gating,

  $v_\theta(x,z) = \sum_{i=1}^E g_i(z) f_i(x)$

• Hybrid Training Objectives:

  • Flow-matching regression ($\mathcal{L}_{FM}$) for continuous action alignment
  • Masked motion token cross-entropy ($\mathcal{L}_{MASK}$) for discrete sequence prediction.
• Unification: All modalities are tokenized for sequence-based multimodal modeling (joint vision, text, state, action streams) (Luo et al., 19 Jan 2026).

5. Robustness Mechanisms

Two explicit deployment stabilizers address sensory shift and real-time latency:

• Manifold-Preserving Gating (MPG):
  • Detects out-of-distribution (OOD) context embeddings $H$ using a sliced-Wasserstein discrepancy between observation and reference action anchors.
  • Computes a gating factor $g = \exp(-D/\tau)$ applied to feature corrections, ensuring robust fallback to reliable priors under OOD or ambiguous observations.
• Universal Async Chunking (UAC):
  • Accommodates diverse robot control frequencies and latencies by adaptively chunking action tokens.
  • Implements a buffer and prefix-locking for real-time execution, maintaining consistent performance regardless of embodiment-specific inference hiccups.
  • Dual-thread rings decouple chunk denoising and execution in deployment (Luo et al., 19 Jan 2026).

These features collectively address the pronounced instability and cross-platform variability endemic to embodied VLA policies.

6. Training Strategy and Task Sampling

Being-H0.5 employs a unified multi-task training regimen:

• Serializing All Modalities: All modalities are framed as sequential token prediction with a joint loss:

$$\mathcal{L} = \lambda_{text}\,\mathcal{L}_{text} + \lambda_{act}\,\mathcal{L}_{act}$$

• Dynamic Task Sampling:
  • Mini-batches interleave human motion, robot trajectories, VQA, planning, and spatial grounding.
  • Motion generation is sampled preferentially, but text generation and continuation are also enforced.
• Slotwise Fine-tuning: Embodiment-specific adaptation via zero-masked slotwise adapters $W_{ESA}[k]$, updating only the active slot set $I_e$ for each target robot.

This comprehensive training approach jointly optimizes across tasks and domains to maximize generalization and sample efficiency (Luo et al., 19 Jan 2026).

7. Evaluation and Empirical Performance

Extensive simulated and real-world evaluation demonstrates high transfer and robustness:

Benchmark	Specialist	Generalist	Prior SoTA
LIBERO (rgb, sim)	98.9%	97.6% (joint)	X-VLA 98.1%
RoboCasa (few-shot, sim)	53.9%	53.3%	π₀.₅ 41.4%
Real robot: Spatial	~85–95%	5–10 pp lower	π₀.₅ ~50–60%
Real robot: Long	~80%	5–10 pp lower	π₀.₅ ~30%
Real robot: Bimanual	~75%	5–10 pp lower	π₀.₅ ~40%

• Platforms: Five distinct robotics platforms spanning 6–31 DoF and various actuation schemes.
• Tasks: Varied task categories (spatial, long-horizon, bimanual, generalization). Blind-evaluated with 20 × 20 configuration-trial grids, binary success.
• Generalization: Zero-shot successes observed on platforms withheld from demonstration data in specific tasks.

This suggests the model’s architecture and learning paradigm generalize effectively under both simulated and real-world cross-embodiment conditions, with notably improved performance compared to prior state-of-the-art and specialist policies (Luo et al., 19 Jan 2026).

---

The Being-H0.5 design advances embodied AI by tightly integrating human physical priors, unified action spaces, and robust multi-modal sequence modeling, establishing new empirical standards for cross-platform skill transfer in robotics.