LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment

Abstract: We propose a new method named LoD-Loc for visual localization in the air. Unlike existing localization algorithms, LoD-Loc does not rely on complex 3D representations and can estimate the pose of an Unmanned Aerial Vehicle (UAV) using a Level-of-Detail (LoD) 3D map. LoD-Loc mainly achieves this goal by aligning the wireframe derived from the LoD projected model with that predicted by the neural network. Specifically, given a coarse pose provided by the UAV sensor, LoD-Loc hierarchically builds a cost volume for uniformly sampled pose hypotheses to describe pose probability distribution and select a pose with maximum probability. Each cost within this volume measures the degree of line alignment between projected and predicted wireframes. LoD-Loc also devises a 6-DoF pose optimization algorithm to refine the previous result with a differentiable Gaussian-Newton method. As no public dataset exists for the studied problem, we collect two datasets with map levels of LoD3.0 and LoD2.0, along with real RGB queries and ground-truth pose annotations. We benchmark our method and demonstrate that LoD-Loc achieves excellent performance, even surpassing current state-of-the-art methods that use textured 3D models for localization. The code and dataset are available at https://victorzoo.github.io/LoD-Loc.github.io/.

• The paper introduces LoD-Loc, a novel method that robustly estimates UAV poses by aligning neural and LoD wireframes using a differentiable Gauss-Newton optimizer.
• It employs a multilevel refinement approach with hierarchical cost volumes and variance-based uncertainty to progressively enhance pose precision.
• Evaluation on the UAVD4L-LoD and Swiss-EPFL datasets shows superior recall metrics compared to textured 3D models, underscoring its scalability and privacy benefits.

Aerial Visual Localization with LoD-Loc

The paper introduces LoD-Loc, a novel method for aerial visual localization utilizing Level-of-Detail (LoD) 3D maps. This approach stands out from existing methods as it requires minimal reliance on complex 3D representations, yet achieves robust pose estimation for UAVs through an innovative alignment of neural wireframes with LoD wireframes.

Methodology Overview

LoD-Loc leverages the simplicity and compactness of LoD 3D maps, which are considerably easier to acquire and store compared to traditional textured 3D models. The method begins with a coarse pose estimate from UAV sensors and refines this through a multilevel process:

• Wireframe Alignment: The alignment of projected wireframes from 3D models with wireframes predicted by a neural network forms the crux of the approach.
• Cost Volume Construction: A hierarchical cost volume is constructed for sampled pose hypotheses, evaluating line alignment between projected and predicted wireframes.
• 6-DoF Pose Optimization: A Gauss-Newton method, modified to be differentiable, is used to refine the poses, optimizing for maximum wireframe alignment.

The multilevel alignment process is guided by variance-based uncertainty measures that refine pose hypotheses, progressively narrowing down to a precise pose solution. This hierarchical refinement is key to achieving high accuracy in the final pose estimation.

Datasets and Evaluation

The authors introduce two new datasets to address the lack of public datasets for LoD-based localization:

1. UAVD4L-LoD: Spanning 2.5 square kilometers, this dataset includes LoD3.0 models generated from a detailed mesh model.
2. Swiss-EPFL: Covering 8.2 square kilometers, this dataset relies on publicly available LoD2.0 models, capturing a variety of urban structures.

The evaluation reveals that LoD-Loc outperforms existing methods, especially those relying on textured 3D models, in recall metrics across multiple thresholds. Even in challenging out-of-trajectory scenarios, the method maintains high performance levels.

Implications and Future Developments

LoD-Loc's use of LoD maps highlights a shift towards more efficient and privacy-preserving localization methods suitable for large-scale deployment. The method's applicability to privacy-sensitive environments is particularly impactful, given its abstraction and minimal data exposure.

Looking forward, this approach paves the way for further exploration into lightweight and secure localization solutions. The interoperability between neural predictions and LoD-based wireframe alignment also suggests potential for broader applications in environments where detailed textures are unavailable or impractical.

Conclusion

LoD-Loc presents a highly promising direction for aerial visual localization by effectively marrying LoD 3D maps with neural network-based wireframe predictions. The innovative use of simpler mapping resources without sacrificing performance marks a significant step forward in UAV localization methods, opening avenues for more efficient, secure, and scalable solutions.