Gravity-aligned Rotation Averaging with Circular Regression

Abstract: Reconstructing a 3D scene from unordered images is pivotal in computer vision and robotics, with applications spanning crowd-sourced mapping and beyond. While global Structure-from-Motion (SfM) techniques are scalable and fast, they often compromise on accuracy. To address this, we introduce a principled approach that integrates gravity direction into the rotation averaging phase of global pipelines, enhancing camera orientation accuracy and reducing the degrees of freedom. This additional information is commonly available in recent consumer devices, such as smartphones, mixed-reality devices and drones, making the proposed method readily accessible. Rooted in circular regression, our algorithm has similar convergence guarantees as linear regression. It also supports scenarios where only a subset of cameras have known gravity. Additionally, we propose a mechanism to refine error-prone gravity. We achieve state-of-the-art accuracy on four large-scale datasets. Particularly, the proposed method improves upon the SfM baseline by 13 AUC@$1^\circ$ points, on average, while running eight times faster. It also outperforms the standard planar pose graph optimization technique by 23 AUC@$1^\circ$ points. The code is at https://github.com/colmap/glomap.

• The paper introduces a novel method that integrates gravity constraints to reduce rotation degrees of freedom, achieving more accurate camera orientation estimation.
• The methodology employs circular regression to effectively handle the periodic nature of rotational data, outperforming traditional linear techniques.
• Experimental results on datasets like EuRoC and KITTI show significant accuracy gains and efficiency improvements, enabling real-time applications in robotics and AR/VR.

Gravity-aligned Rotation Averaging with Circular Regression

This paper presents a novel approach to enhancing rotation averaging in global Structure-from-Motion (SfM) by integrating gravity direction into the process. The authors leverage circular regression to address the inherent periodicity in the optimization problem and achieve more accurate camera orientation estimations.

Key Contributions

The paper's primary contribution is the incorporation of gravity direction as an additional constraint in the rotation averaging problem, reducing the degrees of freedom from 3 to 1. This is made feasible by using the built-in inertial measurement units (IMUs) present in modern consumer devices like smartphones and drones. The approach proposed by the authors is comparable to planar PGO methods but is specifically tailored to utilize gravity data.

Methodology

The methodology introduced employs circular regression to effectively handle the periodic nature of rotational data. Traditional methods often use linear regression techniques that can be ineffective with periodic data. In contrast, the proposed method harnesses these periodic properties for more robust and accurate orientation estimation. This is achieved by iteratively updating both the rotation angles and their periodic offsets, ensuring optimal alignment with the observed gravity directions.

The authors also address scenarios where only a subset of the cameras has known gravity, showcasing the versatility of their method. They propose a stratified approach that separates the problem into subcomponents, handling 1-DoF and 3-DoF rotations accordingly.

Experimental Results

The paper provides experimental validation on several large-scale datasets including EuRoC, KITTI, LaMAR, and 1DSfM. Across these datasets, the proposed method outperforms existing techniques, achieving significant improvements in rotation accuracy. Notably, it shows an average 13 AUC@$1^\circ$ point increase over baseline SfM methods and a 23 point improvement over planar PGO techniques, while being computationally efficient and running eight times faster.

Implications and Future Directions

The integration of gravity direction into rotation averaging not only improves the accuracy of 3D reconstructions but also introduces a computationally efficient solution that scales well with large datasets. This advancement has practical implications for real-time applications in robotics and AR/VR systems where real-time 3D reconstruction is often crucial.

In terms of future directions, the method opens up possibilities for more sophisticated fusion of IMU data with visual information. While the proposed approach handles noisy gravity measurements effectively, further refinements in the sensor fusion algorithms could enhance robustness in environments with more significant sensor drift.

Additionally, the research underscores the potential benefits of circular statistics in computer vision applications, suggesting avenues for exploring other areas where periodic data is prevalent.

Conclusion

The paper makes a significant contribution to the field of computer vision by introducing an efficient and accurate method of rotation averaging that leverages gravity direction via circular regression. It sets a foundation for future research exploring the integration of additional sensory information in 3D reconstruction tasks, potentially extending its applicability and effectiveness across a broader range of devices and scenarios.