EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting

Abstract: With their meaningful geometry and their omnipresence in the 3D world, edges are extremely useful primitives in computer vision. 3D edges comprise of lines and curves, and methods to reconstruct them use either multi-view images or point clouds as input. State-of-the-art image-based methods first learn a 3D edge point cloud then fit 3D edges to it. The edge point cloud is obtained by learning a 3D neural implicit edge field from which the 3D edge points are sampled on a specific level set (0 or 1). However, such methods present two important drawbacks: i) it is not realistic to sample points on exact level sets due to float imprecision and training inaccuracies. Instead, they are sampled within a range of levels so the points do not lie accurately on the 3D edges and require further processing. ii) Such implicit representations are computationally expensive and require long training times. In this paper, we address these two limitations and propose a 3D edge mapping that is simpler, more efficient, and preserves accuracy. Our method learns explicitly the 3D edge points and their edge direction hence bypassing the need for point sampling. It casts a 3D edge point as the center of a 3D Gaussian and the edge direction as the principal axis of the Gaussian. Such a representation has the advantage of being not only geometrically meaningful but also compatible with the efficient training optimization defined in Gaussian Splatting. Results show that the proposed method produces edges as accurate and complete as the state-of-the-art while being an order of magnitude faster. Code is released at https://github.com/kunalchelani/EdgeGaussians.

• The paper presents a novel 3D edge mapping technique using Gaussian splatting that improves accuracy by explicitly representing edge points.
• It overcomes implicit sampling issues and lengthy training times by aligning edge points as centers of 3D Gaussians with defined orientations.
• The method achieves up to 30 times faster runtimes and competitive precision on benchmarks like ABC-NEF and DTU, enabling real-time applications.

EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting

The paper introduces a methodology named EdgeGaussians, which proposes an efficient and explicit representation for 3D edge mapping using Gaussian Splatting. This method addresses the shortcomings of existing state-of-the-art methods that rely on computationally intensive implicit representations and sampling inaccuracies. It establishes a more straightforward, precise, and faster approach for 3D edge reconstruction with a focus on applications in various computer vision tasks.

Introduction

Edges are fundamental visual primitives crucial for various computer vision applications, including mapping, localization, and rendering. Traditional methods for reconstructing 3D edges from images face challenges such as detection inaccuracy and high computational costs. The EdgeGaussians approach aims to overcome these by employing a direct learning method for 3D edges.

The paper highlights two main drawbacks of existing methods: the inefficiency of sampling on exact level sets due to floating-point inaccuracies and long training times involved in learning implicit representations. EdgeGaussians addresses these by using 3D Gaussians to explicitly map edge points, which circumvents the need for point sampling and intensive computations.

Methodology

Gaussian Representation

The proposed method represents 3D edge points as the centers of 3D Gaussians, with the edge directions defined as the principal axes of these Gaussians.

Figure 1: EdgeGaussians: the proposed method learns 3D edge points with an explicit representation where an edge point is cast as a 3D Gaussian centered at that point and the edge direction defines the main orientation of the Gaussian.

Such a representation is not only geometrically meaningful but also allows for efficient training optimizations inspired by Gaussian splatting. The training process is supervised using 2D edge maps generated by pre-existing edge detectors.

Overcoming Occlusion and Sparsity

The methodology accounts for the inherent sparsity and occlusion issues related to 3D edges. Occluded edges are identified, allowing the model to correctly render only the visible parts of edges in the training data as shown below.

Figure 2: Occlusions for 3D edges. The red edges are the occluded edges absent from the supervisory 2D edge maps.

Geometric Constraints

Geometric consistency is enforced by constraining the Gaussians' principal axis to align with their nearest neighbors. This ensures that points on the same edge maintain spatial and directional coherence, which is critical for effective edge clustering and fitting.

Figure 3: The two geometric criteria for the learned 3D Gaussians.

Implementation and Results

Comparative Analysis

EdgeGaussians demonstrates comparable accuracy to the state-of-the-art while achieving significantly faster runtimes—up to 30 times faster than competing methods like EMAP and NEF. Quantitative evaluations on datasets like ABC-NEF and DTU exhibit the method's superior performance in terms of recall and precision at various error thresholds.

Performance on Real-world Data

Qualitative evaluations on real-world datasets indicate that EdgeGaussians effectively captures edges with high completeness and minimal duplications compared to competitors.

Conclusion

EdgeGaussians provides a balance of accuracy, efficiency, and simplicity for 3D edge mapping. Its explicit representation approach circumvents the drawbacks associated with implicit sampling and computational expenses, making it suitable for real-time applications. Future work may involve improving supervision methods to further refine accuracy and broaden application scopes.