See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View

Abstract: UAV Vision-Language Navigation (UAV-VLN) is typically formulated as a holistic search-and-reach problem, where long-range target discovery and final target approach are optimized and evaluated jointly. This formulation makes it difficult to assess a critical capability of aerial embodied agents, namely whether a UAV can accurately ground a visible target and translate vision-language evidence into precise 3D motion once the target enters its field of view. To address this limitation, we introduce UAV-VLN-FOV, a target-visible navigation task that isolates the see-and-reach stage and enables a more diagnostic evaluation of terminal reaching ability. We further propose 3DG-VLN, a vision-language waypoint prediction framework guided by dynamic 3D direction cues to enhance fine-grained visual grounding and spatial direction alignment for precise target reaching. Specifically, 3DG-VLN adaptively processes high-resolution front-view and downward-view observations to preserve fine-grained visual and geometric details for target grounding. It also updates the target-relative direction online during closed-loop navigation, allowing the agent to maintain spatial alignment with the target and reduce accumulated direction drift. To support this task, we construct a dedicated high-resolution benchmark which contains 2,717 trajectories with target-oriented high-level instructions, high-resolution front-view and downward-view egocentric observations, and continuous 3D waypoint annotations. Experiments show that 3DG-VLN outperforms competitive UAV-VLN baselines, achieving a 13.82\% improvement in success rate. Real-world trials further demonstrate the potential of 3DG-VLN for practical see-and-reach navigation. The source code and benchmark are available at https://github.com/xuefanfu/3DG-VLN.

• The paper introduces the UAV-VLN-FOV task that isolates the terminal see-and-reach stage to achieve precise 3D maneuvers under a 10-meter radius.
• It employs high-resolution dual-view processing with dynamic 3D directional cues and online updates to enhance spatial alignment and mitigate drift.
• Experimental results highlight significant improvements in success rates, validated through extensive simulation and real-world deployments.

See-and-Reach Navigation for UAVs: Precise Vision-Language Control Within the Field of View

Motivation and Problem Formulation

The paper "See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View" (2606.20045) identifies a critical gap in the existing UAV vision-language navigation (UAV-VLN) paradigm, which predominantly formulates navigation as a holistic search-and-reach task. The authors argue that conflating long-range target search with terminal approach obfuscates the evaluation of an essential capability: transforming linguistic and visual evidence into precise 3D maneuvers once a target enters the UAV’s field of view. To address this, the UAV-VLN-FOV task is formalized to isolate the see-and-reach stage, emphasizing high-precision waypoint prediction and exact spatial alignment under stringent 10-meter success radius constraints, which are substantially tighter than the commonly adopted 20-meter thresholds. This reformulation is motivated by operational requirements in real-world deployments such as disaster relief, where terminal precision is critical for mission success.

Methodological Innovations

The core contribution, 3DG-VLN, is a vision-language waypoint prediction framework leveraging high-resolution dual-view (front-view and downward-view) observations and dynamic 3D directional cues. The system employs the following technical strategies:

• High-resolution egocentric perception: Unlike traditional approaches using low-resolution imagery, 3DG-VLN adaptively processes high-resolution camera inputs, preserving morphological and contextual details necessary for fine-grained visual grounding, discrimination of distractors, and obstacle awareness.
• Dynamic 3D direction guidance: The navigation pipeline integrates a discretized direction prior (front-left, front, front-right, down), computed in the UAV-centric frame. Critically, the model updates this direction online during inference via object detection (using Grounding DINO) and geometric ray computation, ensuring continual spatial alignment and mitigating drift.
• Egocentric waypoint supervision: Waypoints are predicted as relative displacements in the local UAV frame, providing robust translation and rotation invariance for continuous 3D trajectory planning.
• Integration of multimodal foundation models: The approach fine-tunes Qwen2.5-VL through LoRA adaptation, supporting high-resolution visual-language input and outputting structured body-frame waypoint sequences. The instruction parser (DeepSeek-V3.2) and object detector (Grounding DINO) are invoked to extract semantic targets and relevant geometric priors.

Benchmark Construction and Dataset Properties

To enable rigorous assessment, the authors construct a high-resolution benchmark comprising 2,717 trajectories and 31,878 image-instruction-waypoint tuples spanning 14 diverse environments (rural, town, urban) and 89 target categories. Each trajectory is aligned with concise high-level instructions and continuous 3D waypoints, focusing exclusively on see-and-reach scenarios where the target is visible at the outset. The dataset is partitioned into train, test, unseen-object, and unseen-scene subsets, facilitating fine-grained evaluations of generalization and robustness.

Experimental Evaluation

Comprehensive experiments compare 3DG-VLN with baseline methods including Random, Fixed, vanilla Qwen2.5-VL-7B, and the current SOTA closed-loop waypoint-based approach TravelUAV. Across all test splits, 3DG-VLN exhibits substantial increases in success rate (SR), oracle success rate (OSR), and success-weighted path length (SPL), while correspondingly reducing navigation error (NE). For example, on the standard test set, 3DG-VLN achieves a 13.82% absolute increase in SR over TravelUAV and a 25.66% increase in OSR, underscoring its superior approach and grounding capabilities.

Ablation studies further delineate the contributions of high-resolution processing, directional guidance, and dynamic online direction updates:

• High-resolution input: Models trained on high-resolution images provide significant gains in SR and OSR, validating the necessity for detailed perceptual input in precise navigation.
• Directional guidance: The inclusion of coarse 3D direction cues, especially when updated online, confers marked improvements in spatial alignment and terminal precision.
• Direction updating: Continual direction updates throughout the trajectory prevent drift and misalignment, especially evident in comparison with static or outdated direction strategies.

Real-world experiments corroborate simulation findings, demonstrating the agent's robustness and accuracy in translating high-level instructions and real-time observations into precise 3D maneuvers under environmental variations and partial observations. The system demonstrates successful navigation to objects and locations with adherence to the 10-meter success radius.

Theoretical and Practical Implications

Isolating the see-and-reach stage reframes UAV-VLN evaluation, enabling systematic diagnosis of terminal precision failures and revealing gaps in visual grounding under realistic operational constraints. The approach offers formal advances in closed-loop continuous motion planning, leverage of foundation multimodal models, and adaptive spatial alignment. Practically, the method enables UAVs to perform precise delivery, inspection, and interaction tasks in high-stakes domains without reliance on coarse search-and-reach metrics, aligning navigation policy fidelity with real deployment criteria.

The benchmark provides a testbed for future research in fine-grained embodied navigation, targeting the transition from semantic exploration to geometric alignment and control. By advocating for more stringent success metrics, the paper raises the bar for evaluation in aerial embodied intelligence and prompts deeper exploration of model adaptation, multimodal grounding, and spatial reasoning.

Future Directions

The current reliance on external object detectors (Grounding DINO) introduces sensitivity to detection noise; integrating detection, grounding, and planning into a joint end-to-end learned model remains an open direction. Further exploration of multimodal fusion, real-time adaptation, and memory-based spatial reasoning architectures may yield stronger performance and generalization. The UAV-VLN-FOV dataset sets a foundation for the broader study of field-of-view-constrained multimodal navigation and interaction in aerial systems.

Conclusion

This work defines and addresses the UAV-VLN-FOV task, emphasizing precision navigation within the field of view and establishing rigorous evaluation protocols. The proposed 3DG-VLN framework delivers SOTA performance by combining high-resolution dual-view processing with online spatial direction guidance, validated by both simulation and real-world deployment. The new benchmark and methodology offer significant foundations and challenges for future research in vision-language navigation and aerial embodied intelligence (2606.20045).