EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$

Abstract: While Multimodal LLMs (MLLMs) have demonstrated exceptional capabilities in standard visual understanding, adapting them for active visual search in 360$^\circ$ panoramic environments exposes fundamental limitations. Specifically, standard MLLMs struggle to effectively model inherent panoramic properties, such as severe polar distortion and continuous cylindrical topologies, which significantly degrades target detection accuracy. Consequently, existing panoramic search methods attempt to compensate by relying heavily on fragmented local viewpoints. Burdened by rigid initialization and a lack of global panoramic priors, these approaches suffer from myopic, inefficient exploration and struggle with robust error recovery when targets are out of view. To overcome these challenges, we propose EAGLE-360, a novel Embodied Active Global-to-Local Exploration framework. Rather than performing exhaustive local searches, EAGLE-360 leverages global priors to establish an initial holistic perspective, iteratively reasoning and progressively narrowing the search space. Architecturally, we adapt RoPE Rolling, a coordinate-shifting positional encoding mechanism, to seamlessly model the continuous topologies of panoramas. To facilitate this paradigm, we construct the large-scale EAGLE-360 dataset, comprising 14,000+ 4K panoramas and 70,000+ rounds of high-quality VQA dialogues. By employing a training pipeline that integrates Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization (GRPO), we effectively elicit complex spatial reasoning and tool-calling capabilities. Extensive experiments demonstrate that EAGLE-360 establishes a new state-of-the-art for 360$^\circ$ visual search, achieving nearly an 8-fold increase in accuracy over the base model while significantly enhancing exploration efficiency.

• The paper presents a novel global-to-local framework that uses spherical-aware perception and RoPE Rolling to overcome local reasoning deficits in 360° panoramas.
• It introduces a two-stage perception-action loop with tool-augmented exploration, enabling iterative camera adjustments for precise object localization.
• Experimental results show nearly 8-fold improvement over baselines with 64.44% accuracy, validating its robustness across varied panoramic settings.

EAGLE-360: Embodied Active Global-to-Local Exploration in 360° Panoramas

Introduction and Motivations

The challenge of embodied active visual search in panoramic environments exposes significant architectural and cognitive deficits in MLLMs that were originally engineered for planar inputs. Standard panoramic exploration frameworks suffer from local reasoning, myopic iterative search, and vulnerability to geometric distortions and topological discontinuities inherent to equirectangular imaging. The EAGLE-360 framework directly addresses these deficiencies by formalizing panoramic object search as a global-to-local reasoning problem that leverages spherical geometry-aware perception mechanisms and novel reinforcement learning adaptations.

Problem Formulation and Panoramic Modeling

The search process is formalized as a POMDP where the agent must localize a described object within a $360^\circ$ equirectangular panorama $I_\text{pano}$, outputting a bounding field of view (BFoV) parameterized on the sphere by center azimuth $\theta_c$, elevation $\phi_c$, and FoV extents $(f_h, f_v)$. A success is signaled if the great-circle distance between the predicted and ground-truth center points is within half the bounding box's diagonal angle. This spherical evaluation eliminates planar projection artifacts and enforces geometric rigor in the localization process.

The core system pipeline incorporates a two-stage perception-action loop:

1. Global Perception: The agent performs holistic panoramic reasoning using an adapted rotary positional encoding—RoPE Rolling—which allows transformer-based MLLMs to resolve the wrap-around cylinder topology and maintain feature continuity at left/right boundaries.
2. Local Refinement via Tool-Augmented Exploration: The agent iteratively adjusts a virtual camera over the sphere, invoking a perspective projection tool to generate locally undistorted views and recursively narrows the FoV, articulating evidence at each step within a chain-of-thought (CoT) reasoning trace, ultimately emitting the target's precise BFoV.

   Figure 1: The overall pipeline of the proposed EAGLE-360 framework, illustrating the transition from global panorama perception with RoPE Rolling to iterative, tool-augmented local exploration.

Dataset Composition and Augmentation

The EAGLE-360 dataset encompasses over 14,000 high-resolution panoramas (4K–8K) from heterogeneous environments, yielding 70,000+ multi-turn spatial-reasoning dialogues. Images are sampled from Kuula, Matterport3D, and 2D-3D-Semantics, providing extensive coverage of residential, public, and in-the-wild scene topologies.

To support robust pano-coordination, four-direction horizontal rotation augmentation is employed, generating uniform azimuthal object distributions and enforcing rotational invariance. Each annotation is validated for referential uniqueness, visual clarity, and spatial discriminability through a combination of automated filtering (GroundingDINO, GPT-4o-mini) and rigorous expert curation.

Figure 2: A training data sample from Kuula, illustrating scene complexity and spatial diversity.

Figure 3: A training data sample from Matterport3D, highlighting dense indoor semantic context.

Figure 4: A training data sample from 2D-3D-S, showing structured academic environments and unique object arrangements.

Architecture: RoPE Rolling and Active Tool Use

Classical RoPE encodings functionally sever the cylindrical nature of panoramic images at the left/right boundary, disrupting attention coherence. The RoPE Rolling mechanism offset-rolls horizontal coordinates on a per-head basis in the transformer, ensuring each vertical column serves as the contextual center for at least one attention head. This enables seamless attention and consistent positional encoding across the panorama's boundary, which is critical for panoramic exploration and object localization.

Following the initial global estimation, the framework enters a tool-calling loop, wherein each "thought-action-observation" cycle involves camera redirection and narrowed FoV projection, with fine-grained spatial reasoning accumulating iteratively in the context window. This approach directly overcomes the myopia of earlier fixed-viewport methods.

Training Paradigm: SFT and Group Relative Policy Optimization

Training is staged in two phases:

• Supervised Fine-Tuning (SFT): Aligns the base MLLM to the multi-instruction format, enforcing proper tool use, FoV specification, and spatial reasoning articulation.
• Group Relative Policy Optimization (GRPO): Reinforces effective policies in ultralong dialogue contexts by computing trajectory advantages relative to group means, rather than absolute values, avoiding value network dependency and enabling greater stability in sparse-reward or highly multimodal settings.

A composite reward configures answer precision (discrete ACC, GCD), tool-use efficiency, format compliance, and length penalties, with empirical ablation demonstrating that discrete accuracy-based rewards maximize final localization success rates.

Quantitative Results and Multi-Turn Exploration Dynamics

EAGLE-360 achieves 64.44% accuracy—nearly 8-fold improvement over the strongest base model—and reduces great-circle localization error to 16.89°, establishing new SOTA on the EAGLE-360 benchmark. Notably, performance in challenging azimuthal regions ("Back", "Left", "Right") reaches above 55%. On out-of-distribution H*Bench transfer, EAGLE-360 attains 56.1 overall (74.8 on Humanoid Object Search), substantially outperforming proprietary (Gemini-2.5-Pro) and open-source baselines.

Figure 5: Step-wise submission and accuracy, showing delayed answer commitments and higher spatial precision through GRPO-driven exploration.

Qualitative Visualization and Case Analysis

Inference cases illustrate that EAGLE-360 rapidly converges from a global hypothesis towards a narrow, target-centered FoV, avoiding the pitfalls of false localization from initial misorientations. The CoT framework demonstrates actionable spatial reasoning, evidence articulation, and adaptive path correction—essential for robust embodied search in unconstrained settings.

Figure 6: A representative inference case demonstrating coarse-to-fine spatial convergence in EAGLE-360.

Figure 7: Inference trajectory visualizing tool-calling steps, projections, and the grounding of the correct object.

Figure 8: Successful object localization in a complex scene, evidencing EAGLE-360’s resilience to challenging layouts.

Discussion: Implications, Generalization, and Limitations

Theoretical Implications: EAGLE-360 structurally uncouples panoramic search from the pitfalls of local egocentric initialization, instead framing the problem as global hypothesis refinement over spherical observation. RoPE Rolling stands as a principled approach for encoding non-Euclidean image topologies within standard transformer backbones, bridging geometric AI with scalable self-attention architectures.

Practical Deployability: The framework’s explicit tool-use structure and chain-of-thought protocol are highly amenable to real-world embodied agents and robotics, provided inference-time context optimization (e.g., cache compression or structured memory) can address transformer scaling limitations. Polar-region distortion and extremely verbose rollout histories remain active challenges.

Future Directions: The paper motivates extension to dynamic (video-based) 4D environments, integration with privacy-preserving distributed training, and development of true spherical vision backbones to standardize performance in all spatial regimes.

Conclusion

EAGLE-360 establishes a robust and generalizable paradigm for panoramic active visual search by integrating cylindrical topology-aware positional modeling, global-to-local active reasoning, and reinforcement learning within large multimodal LLMs. The rigorously-constructed EAGLE-360 dataset and ablation analysis validate the practical significance of the approach. These advances open new directions for embodied AI operating in unstructured panoramic spaces and provide a blueprint for panoramic-aware architectures applicable to both static and dynamic multimodal reasoning tasks (2607.02479).