HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps

Abstract: Visual localization on standard-definition (SD) maps has emerged as a promising low-cost and scalable solution for autonomous driving. However, existing regression-based approaches often overlook inherent geometric priors, resulting in suboptimal training efficiency and limited localization accuracy. In this paper, we propose a novel homography-guided pose estimator network for fine-grained visual localization between multi-view images and standard-definition (SD) maps. We construct input pairs that satisfy a homography constraint by projecting ground-view features into the BEV domain and enforcing semantic alignment with map features. Then we leverage homography relationships to guide feature fusion and restrict the pose outputs to a valid feasible region, which significantly improves training efficiency and localization accuracy compared to prior methods relying on attention-based fusion and direct 3-DoF pose regression. To the best of our knowledge, this is the first work to unify BEV semantic reasoning with homography learning for image-to-map localization. Furthermore, by explicitly modeling homography transformations, the proposed framework naturally supports cross-resolution inputs, enhancing model flexibility. Extensive experiments on the nuScenes dataset demonstrate that our approach significantly outperforms existing state-of-the-art visual localization methods. Code and pretrained models will be publicly released to foster future research.

• The paper introduces a homography-guided pose estimator that jointly optimizes homography estimation and camera pose regression to improve localization accuracy.
• It integrates geometric alignment cues with multi-level feature fusion to enhance robustness against occlusion, illumination, and scale variations.
• Extensive experiments and ablation studies demonstrate significant gains in translation and rotation metrics on SD maps for real-time navigation.

HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps

Problem Context and Motivation

Fine-grained visual localization on semantic-dense (SD) maps is central to navigation and autonomous driving tasks, where highly accurate camera pose estimation is required in complex urban environments that exhibit frequent viewpoint and appearance changes. Traditional map-based visual localization pipelines, which rely on feature matching and coarse candidate retrieval, struggle to generalize to open-set conditions, particularly with perspective shifts, missing environment details, or significant domain gaps between map and query imagery. Recent advancements such as neural matching and direct pose regression have pushed the field, but challenges persist in closing the domain disparity and achieving robust, fine-grained pose estimation, especially under challenging illumination and occlusion.

Core Contributions and Methodology

The HOLO framework introduces a homography-guided pose estimator network specifically designed to address cross-view fine-grained visual localization between camera images and SD maps. The principal innovations encompass the following:

• Homography-Conditioned Pose Network: Unlike prior works that regress pose directly or leverage only local feature correlations, HOLO utilizes a deep homography estimation module to parameterize the geometric relation between ground-level images and overhead SD maps. This module is integrated into a pose estimation backbone, allowing geometric cues to guide camera pose prediction.
• End-to-End Architecture: The network is trained to jointly optimize homography estimation and camera pose regression, directly minimizing pose errors while benefiting from geometric alignment signals. The dual-branch approach allows for effective learning of cross-modal correspondences despite appearance and scale variations.
• Refined Pose Regression: By leveraging the estimated homography, pose regression is effectively regularized, resulting in improved accuracy and robustness. The architecture further incorporates multi-level fusion and local-global feature aggregation to support challenging cases with strong occlusion or poor map-image overlap.

Empirical Evaluation and Results

The experimental evaluation demonstrates that HOLO achieves state-of-the-art accuracy on publicly available SD map localization benchmarks and large-scale real-world datasets. Quantitative results reveal significant gains in both translation and rotation error metrics compared to competitive baselines, including neural matching-based approaches and regression-only models. The integration of homography guidance consistently improves pose outlier rejection rates and delivers superior fine-grained localization, especially in low-texture or urban canyon scenarios where classical techniques deteriorate.

The paper also includes thorough ablation studies substantiating the impact of homography conditioning and feature fusion. Runtime efficiency demonstrates that the method scales to real-time operation, which is crucial for practical autonomous driving and robotic navigation pipelines.

Implications and Future Directions

The introduction of HOLO provides a substantive advancement in SD map-based camera localization through explicit geometric conditioning. The results imply that explicit modeling of planar geometric transformations, even in complex cross-view settings, remains highly beneficial when properly integrated with modern deep learning pipelines. Practically, this enables robust map-relative localization under broad environmental and temporal variations, reducing reliance on dense ground-truth annotations and pushing toward broader deployment in open-world settings.

Theoretically, the framework bridges the gap between classic photogrammetric camera resectioning (using homographies) and current data-driven multimodal learning, opening avenues for hybrid geometric-learning systems. Future directions may involve extending the framework to more generalized homography families (e.g., handling scale, non-planarity), incorporating active uncertainty estimation, and adapting the network for self-supervised or few-shot learning scenarios to reduce labeling costs. Broader integration with vectorized map representations and multi-modal sensor fusion is also likely to yield further gains in localization robustness and generalizability.

Conclusion

HOLO establishes a homography-guided paradigm for visual pose estimation on SD maps, effectively combining geometric parameterization with deep learning-based feature correlation and regression. The approach sets new benchmarks for fine-grained localization accuracy and robustness, providing a scalable architecture for real-world navigation systems. The work underscores the value of homography-based geometric cues in deep pose estimation and lays a foundation for future research into robust, domain-invariant localization across diverse urban and open-set scenarios.