AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration

Abstract: In recent years, multimodal LLMs (MLLMs) have shown strong potential for embodied intelligence, yet their ability to maintain geometrically consistent spatial understanding across heterogeneous views remains under-evaluated. Existing benchmarks largely focus on single-agent, single-view perception, leaving a gap in the systematic assessment of collaborative air-ground settings, where multi-scale observations are complementary but introduce scale mismatch, asymmetric occlusion, and reference-frame inconsistencies. We present AirGroundBench, a diagnostic benchmark for evaluating multi-view spatial intelligence in heterogeneous UAV-UGV collaboration. AirGroundBench is built from 11 high-fidelity simulated environments with 1,021 synchronized air-ground observation pairs, yielding approximately 62,000 dual-view, four-option single-choice visual question answering instances and 115 closed-loop vision-language navigation episodes. It covers 10 task types organized into four progressively demanding capability dimensions: spatial perception, cross-view alignment, spatial transformation and reasoning, and embodied decision-making. To support geometry-grounded evaluation and analysis, we provide structured spatial annotations, including cross-view object identities and metric 2D and 3D bounding boxes. Evaluations of 13 representative MLLMs under UAV-only, UGV-only, and dual-view input settings reveal consistent bottlenecks: models perform relatively well on spatial perception but struggle with cross-view alignment and transformation-intensive reasoning, and these deficits propagate to sequential decision-making in vision-language navigation. Although dual-view inputs provide measurable gains over single-view variants, a persistent gap from human performance remains, highlighting geometric consistency as a key limitation of current embodied MLLMs.

• The paper introduces a novel benchmark, AirGroundBench, to evaluate MLLMs’ capability in cross-view spatial reasoning under heterogeneous air-ground collaboration.
• It systematically analyzes performance declines from perception to alignment and navigation, highlighting critical challenges in maintaining geometric consistency.
• Results show dual-view inputs boost performance, yet current models lag behind human-level embodied decision-making, urging the development of explicit geometric grounding mechanisms.

AirGroundBench: Assessing Multi-View Spatial Intelligence in Embodied AI

Motivation and Benchmark Design

The "AirGroundBench" benchmark (2606.28049) targets a well-defined gap in the assessment of multimodal LLMs (MLLMs): multi-view spatial reasoning under heterogeneous air-ground embodied collaboration. Unlike prior benchmarks, which predominantly focus on either single-agent, single-view perception or homogeneous camera setups, AirGroundBench is constructed to systematically probe a MLLM’s ability to form and maintain geometrically consistent spatial representations across complementary aerial (UAV) and ground (UGV) perspectives.

Heterogeneous air-ground collaborations naturally introduce scale mismatch, occlusion asymmetry, and severe reference-frame inconsistencies which are not well-captured in other spatial intelligence or vision-language navigation (VLN) datasets. AirGroundBench consists of high-fidelity simulation environments spanning urban, rural, and open wild spaces, providing 1,021 tightly synchronized UAV-UGV dual-view observation pairs, 62,000 multi-view 4-way VQA instances, and 115 closed-loop VLN episodes. All instances come with rich geometry-grounded annotations, including metric 2D/3D bounding boxes and cross-view object identities.

Task Taxonomy and Construction

AirGroundBench decomposes embodied spatial intelligence into four capability dimensions:

1. Spatial Perception: Recognizing global layouts and object-level properties under multi-view, scale, and semantic variation.
2. Cross-View Alignment: Establishing geometrically correct correspondences, handling scale alignment, and binding entities across heterogeneous perspectives.
3. Spatial Transformation and Reasoning: Solving tasks that require spatial transformations, complex deduction, viewpoint re-mapping, and counterfactual scenario reasoning.
4. Embodied Decision-Making: Sequentially integrating spatial knowledge over time for closed-loop vision-language navigation.

The benchmark uses a semi-automatic generation pipeline to produce tasks grounded by simulation state, cross-checked for ambiguity and spatial grounding. UAV viewpoints are sampled relative to UGV pose in the local NED frame to ensure systematic cross-view geometric relationships.

Experimental Evaluation and Key Results

Thirteen state-of-the-art MLLMs (including proprietary and open-source models) are evaluated in zero-shot mode under three input regimes: UAV-only, UGV-only, and dual-view. Human performance and random baselines are included as reference points. Metrics include classification accuracy for each VQA task and standard embodied navigation metrics—Success Rate (SR), Success weighted by Path Length (SPL), and Navigation Error (NE)—for VLN episodes.

Strong Numerical Findings and Capability Bottlenecks

• Dual-view input provides a significant boost over either UAV-only or UGV-only, empirically confirming the complementary nature of air-ground observations.
• Even the best MLLMs achieve only mid-50% average accuracy on VQA tasks, with the human reference at ~81%. For embodied navigation, the best model achieves VLN SR < 40% versus human ~90%.
• There is a steep degradation in performance from perception (scene/object recognition, ~87-97%) to cross-view alignment and spatial transformation (down to 44-49%), with the largest drop in tasks requiring geometric consistency (e.g., metric calibration, identity binding).
• Error accumulation in closed-loop navigation is pronounced; small spatial inconsistencies in early perception/alignments propagate, leading to catastrophic failures in sequential decision-making.
• Environment generalization is limited: Scenes with less structural anchoring (open wild environments) induce larger performance degradation and variance, underscoring MLLM reliance on persistent geometric cues.

Implications for Embodied AI and Future Directions

AirGroundBench establishes geometric consistency as the central bottleneck for spatial intelligence in current MLLMs operating under multi-view heterogeneity. While perception from rich multimodal data is handled reasonably well, present models are fundamentally limited in their ability to maintain metric-consistent, cross-view, and temporally persistent spatial representations. This presents inherent challenges for applications in cooperative robotics, multi-agent navigation/planning, and any embodied setting that must reason across severely disparate perspectives.

From a theoretical standpoint, these findings call for more explicit geometric grounding mechanisms within MLLMs, beyond the current paradigms of implicit learning from web-scale image-text pairs. Approaches such as architecture-level SE(3)-equivariant modeling, learned scene graphs, and explicit coordinate-frame tracking may be required. Practically, advancements in data curation for training (e.g., synthetic multi-view scenes, densely annotated with ground truth geometries) and cross-disciplinary integration of spatial cognition models could drive improvement.

Conclusion

AirGroundBench provides a rigorous and structured framework to diagnose and quantify the limitations of multimodal large models in geometric, cross-view spatial reasoning and embodied decision making under heterogeneous air-ground collaboration. The consistent capability hierarchy revealed—perception >> alignment > reasoning >> navigation—underscores both the progress in and current deficits of MLLM-based embodied AI. This benchmark is poised to be a key reference point for driving advances in spatially aware multimodal intelligence, robust multi-agent collaboration, and practical deployed embodied systems.