Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations

Abstract: Vision-language-action (VLA) models open a new path toward intuitive robot control by directly linking perception, language, and action in a single end-to-end framework. Yet for UAVs, practical adoption remains difficult because existing solutions are either computationally heavy or insufficiently capable in complex environments. In this work, we propose a practical expert-distillation pipeline (Exp2VLA) for language-conditioned drone navigation. The core idea is to distill expert behavior, obtained from reinforcement learning, teleoperation, or other controllers, into training data that can be used to fine-tune compact VLA models. This allows existing control strategies to be transferred into a unified language-guided navigation model, reducing manual system integration and lowering the barrier for deploying new robot behaviors. Experiments in both sim-to-sim and simulation-in-the-loop settings across multi-object scenes show that the fine-tuned models can handle varied semantic commands and generalize to unseen target compositions. The proposed framework demonstrates how expert-policy distillation can help mechatronic systems move from specialized control modules toward more flexible and reusable robot intelligence.

• The paper presents Exp2VLA, a pipeline that distills expert UAV control policies from reinforcement learning and teleoperation into language-conditioned models.
• It leverages a mid-scale π0.5 VLA architecture with frozen vision encoders to achieve 84.1% success in single-object and 65% in multi-object settings.
• The framework supports real-time, closed-loop sim-to-real deployment, ensuring efficient operation on resource-constrained edge devices.

Exp2VLA: Expert-to-Vision-Language-Action Policy Distillation for UAV Navigation

Introduction

The challenge of integrating vision-language-action (VLA) models with unmanned aerial vehicles (UAVs) lies in the stringent resource constraints of edge devices and the scarcity of domain-specific, expert demonstration data. The paper "Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations" (2607.03146) introduces Exp2VLA, a systematic pipeline that distills expert control policies—obtained via reinforcement learning or teleoperation—into training data for VLA fine-tuning. The resulting framework empowers mid-scale VLA models (notably the $\pi_{0.5}$ architecture) to perform real-time, language-conditioned navigation in multi-object environments, bridging the gap between high-capacity but impractical models and resource-efficient but weak systems.

Figure 1: Overview of the Exp2VLA pipeline, illustrating expert data collection, dataset formation, VLA fine-tuning, and closed-loop deployment in both sim-to-sim and SITL contexts.

Methodology

Exp2VLA is designed as a versatile and modular pipeline comprising three primary stages: expert demonstration collection, dataset structuring, and VLA fine-tuning through imitation learning.

Expert demonstrations are generated using PPO-trained policies within Isaac Lab, yielding a comprehensive set of state-action pairs annotated with high-resolution RGB images and explicit language instructions. All data are serialized to the LeRobot schema, ensuring cross-compatibility with downstream VLA training codebases. The framework supports both single-object and heterogeneous multi-object scenes, accommodating varied spatial configurations and combinatorial generalization tasks.

The policy fine-tuning leverages two VLA backbones: the 4B-parameter $\pi_{0.5}$ (utilizing PaliGemma-3B vision-language backbone and a 300M Flow Matching control head) and SmolVLA (0.45B parameters). For tractability on consumer hardware, only the action prediction modules are updated—vision encoders are frozen—using BF16 precision and gradient checkpointing. Training is conducted for 60,000 steps with losses and optimization stability carefully monitored.

Figure 2: The Exp2VLA pipeline comprises demonstration collection, LeRobot conversion, VLA imitation learning, and closed-loop sim-to-sim evaluation.

Experimental Setup and Evaluation

Experiments are performed in Isaac Lab, with the drone navigating via continuous velocity commands derived from first-person camera feeds and tokenized natural language instructions. Multi-action chunking is used to reduce real-time inference overhead: the policy predicts action sequences executed over a chunk, preserving real-time operation with minimal lag.

Three environments are constructed for comprehensive assessment: (i) a canonical single-object scenario, (ii) a multi-object heterogeneous scene (red cube, green cone, blue cylinder), and (iii) a color-permuted out-of-distribution evaluation set with novel color-shape assignments. Evaluation metrics focus on strict task success rates (within a 0.6m acceptance radius), closed-loop trajectory accuracy, and zero-shot generalization to unseen object-language pairings.

Figure 4: Training environments, including single-object and multi-object setups; drone trajectories in 3D and projected planes demonstrate convergence to language-specified targets.

Results

The $\pi_{0.5}$-based Exp2VLA controller achieves 84.1% success in the single-object setting and 65% mean success over the three-object multi-target setting. On color-permuted, out-of-distribution objects, the model retains 51.7% average success rate, establishing robust compositional generalization. In contrast, SmolVLA achieves only 46.6% in the single-object regime and fails ($\sim$15% success) with combinatorial distractors, indicating inadequate capacity for complex multimodal reasoning.

Latency and VRAM profiling show that while SmolVLA can halve the memory footprint, real-time deployment (23–26 ms inference-loop latency) is feasible only with $\pi_{0.5}$ when balancing semantic fidelity and resource usage.

The closed-loop SITL deployment demonstrates that policies distilled via Exp2VLA not only function in high-fidelity simulation but also transfer seamlessly to real drones via off-board inference and ROS2 actuation, closing a significant engineering loop between demonstration, model adaptation, and autonomous vision-language-action deployment.

Figure 3: SITL pipeline and real-world transfer—sim-to-sim and SITL trajectories for $\pi_{0.5}$ show consistent language-conditioned target acquisition on real flight hardware.

Discussion

The present results contradict claims that only massive VLA architectures (e.g., OpenVLA-7B) are viable for generalizable embodied navigation. Instead, mid-scale VLAs, when distilled via Exp2VLA, realize meaningful zero-shot reasoning and robust, closed-loop control under stringent computational budgets. The bottleneck for further miniaturization lies with the vision backbone's semantic capacity, not merely with parameter count or action head size.

Exp2VLA provides a reproducible, dataset-driven solution for transferring any expert controller into a language-guided, end-to-end reactive policy. Importantly, multi-task language grounding during fine-tuning is essential for avoiding overfitting and for fostering cross-compositional scene understanding.

Implications and Future Directions

Practically, this work lowers the barrier for deploying VLA-controlled drones in real-world, cluttered environments with natural instruction interfaces. The pipeline's modularity encourages re-use across new robotic platforms and tasks, subject only to the availability of expert rollouts.

Theoretically, the Exp2VLA pipeline encourages a revisitation of the scale-efficiency frontier in VLA architectures. The results motivate future research into (1) parameter-efficient adaptation strategies (e.g., PEFT, adapters), (2) disentanglement of visual and linguistic scene grounding in resource-constrained settings, and (3) standardized environments and evaluation protocols for VLA benchmarking in aerial robotics.

Conclusion

Exp2VLA constitutes a significant progression for language-conditioned UAV control: it demonstrates, through quantitative and qualitative evidence, that expert policy distillation into mid-scale VLA backbones yields real-world competent, language-driven navigation systems even in the absence of domain-scale demonstration corpora. The pipeline achieves strong in-distribution control, robust zero-shot generalization, and real-time closed-loop deployment on physical UAV hardware. These findings underscore the feasibility of embedding semantic reasoning in computationally-limited aerial platforms and outline a clear path for resource-aware, VLA-driven robotics.