2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Reconstruction

Abstract: 2D Gaussian Splatting has recently emerged as a significant method in 3D reconstruction, enabling novel view synthesis and geometry reconstruction simultaneously. While the well-known Gaussian kernel is broadly used, its lack of anisotropy and deformation ability leads to dim and vague edges at object silhouettes, limiting the reconstruction quality of current Gaussian splatting methods. To enhance the representation power, we draw inspiration from quantum physics and propose to use the Gaussian-Hermite kernel as the new primitive in Gaussian splatting. The new kernel takes a unified mathematical form and extends the Gaussian function, which serves as the zero-rank term in the updated formulation. Our experiments demonstrate the extraordinary performance of Gaussian-Hermite kernel in both geometry reconstruction and novel-view synthesis tasks. The proposed kernel outperforms traditional Gaussian Splatting kernels, showcasing its potential for high-quality 3D reconstruction and rendering.

• The paper introduces 2DGH, which augments traditional Gaussian splatting with Hermite polynomials to capture complex 3D structures.
• It proposes a novel Gaussian-like activation function to ensure stable opacity and numerical robustness in rendering.
• Experimental results on Synthetic NeRF, DTU, and Mip-NeRF 360 datasets reveal state-of-the-art metrics in SSIM, PSNR, and Chamfer Distance.

2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Reconstruction

The paper "2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Reconstruction" by Ruihan Yu, Tianyu Huang, Jingwang Ling, and Feng Xu from Tsinghua University delivers a significant contribution to the domain of 3D reconstruction and novel-view synthesis (NVS).

Summary

This paper introduces a novel method named 2D Gaussian-Hermite Splatting (2DGH). The authors propose augmenting the traditional 2D Gaussian splatting (2DGS) kernel with Gaussian-Hermite polynomials to enhance the representational capacity of Gaussian splatting techniques. Specifically, they modulate the Gaussian functions with Hermite series to form a new kernel that can better handle complex structures and sharp discontinuities in 3D scenes. The work is motivated by observed limitations in the current Gaussian splatting methods, particularly their inability to express anisotropic and fine structural details effectively.

Key Contributions

1. Introduction of Gaussian-Hermite Kernels:
   • The authors introduce a family of Gaussian-Hermite kernels that significantly extend the representational power of Gaussian functions.
   • They mathematically demonstrate that the original Gaussian kernel is a special case within their formulation, corresponding to the zero-rank term of the Hermite series.
2. New Activation Function:
   • A novel Gaussian-like activation function is proposed to handle the large and potentially negative coefficients resulting from high-rank Gaussian-Hermite functions.
   • This activation function ensures that the opacity values remain within a valid range, maintaining physical plausibility and numerical stability.
3. Implementation and Experimental Validation:
   • The paper modifies the original 2DGS CUDA kernel to accommodate Gaussian-Hermite polynomials and the new activation function.
   • Experiments on Synthetic NeRF, DTU, and Mip-NeRF 360 datasets demonstrate superior performance concerning both geometry reconstruction and rendering quality. Metrics such as SSIM, PSNR, LPIPS, and bidirectional Chamfer Distance were utilized for comprehensive evaluation.

Experimental Findings

The experiments validate the significant improvement in both geometry reconstruction and rendering quality. Specifically, on the Synthetic NeRF dataset, the 2DGH method achieves state-of-the-art performance metrics, surpassing traditional 2DGS and 2DGES methods. Quantitative comparisons highlight:

• Rendering Quality:
  • 2DGH demonstrates better SSIM, PSNR, and LPIPS values compared to both 2DGS and 2DGES.
• Geometry Reconstruction:
  • 2DGH shows a marked improvement in Chamfer Distance, indicating its enhanced capability in reconstructing fine and sharp edges.

Practical and Theoretical Implications

The proposed 2DGH framework offers several practical advantages. By improving the fidelity of geometric and rendering results, this method can be pivotal in applications requiring high precision, such as virtual reality, augmented reality, and digital heritage preservation. Additionally, the mathematical formulation of Gaussian-Hermite splatting aligns closely with principles from quantum physics, setting a theoretical foundation for future cross-disciplinary research.

Theoretical implications suggest that the broader family of Gaussian-Hermite functions can serve as a powerful basis for reconstructing detailed 3D structures. This bridges some of the gaps between mesh-based and splatting-based representations, potentially facilitating more robust and versatile representations of 3D geometry.

Future Directions

Considering the capabilities demonstrated by 2DGH, future work could explore the following aspects:

1. Extending to 3D Gaussian Splatting:
   • Adapting the Gaussian-Hermite polynomial framework to 3D Gaussian splatting may further enhance the rendering and reconstruction capacities for volumetric data.
2. Integrating with Deep Learning Frameworks:
   • The representational advantages of 2DGH can be integrated with deep learning approaches for improved end-to-end learnable models in 3D vision applications.
3. Exploration of Other Polynomial Bases:
   • Beyond Hermite polynomials, exploring other complete sets of basis functions, such as Fourier-based representations, may yield further improvements in splatting techniques.

In conclusion, the 2DGH method stands as a robust enhancement to traditional Gaussian splatting approaches, offering substantial improvements in 3D geometry reconstruction and rendering quality. Its foundational approach and practical implementations underscore its potential to influence future innovations in computer vision and graphics.