DehazeGS: Seeing Through Fog with 3D Gaussian Splatting

Abstract: Current novel view synthesis tasks primarily rely on high-quality and clear images. However, in foggy scenes, scattering and attenuation can significantly degrade the reconstruction and rendering quality. Although NeRF-based dehazing reconstruction algorithms have been developed, their use of deep fully connected neural networks and per-ray sampling strategies leads to high computational costs. Moreover, NeRF's implicit representation struggles to recover fine details from hazy scenes. In contrast, recent advancements in 3D Gaussian Splatting achieve high-quality 3D scene reconstruction by explicitly modeling point clouds into 3D Gaussians. In this paper, we propose leveraging the explicit Gaussian representation to explain the foggy image formation process through a physically accurate forward rendering process. We introduce DehazeGS, a method capable of decomposing and rendering a fog-free background from participating media using only muti-view foggy images as input. We model the transmission within each Gaussian distribution to simulate the formation of fog. During this process, we jointly learn the atmospheric light and scattering coefficient while optimizing the Gaussian representation of the hazy scene. In the inference stage, we eliminate the effects of scattering and attenuation on the Gaussians and directly project them onto a 2D plane to obtain a clear view. Experiments on both synthetic and real-world foggy datasets demonstrate that DehazeGS achieves state-of-the-art performance in terms of both rendering quality and computational efficiency. visualizations are available at https://dehazegs.github.io/

• The paper introduces DehazeGS, the first differentiable physics-based 3D Gaussian Splatting dehazing model for view synthesis from multi-view foggy images.
• DehazeGS leverages explicit Gaussian representations to model atmospheric scattering and incorporates prior information to achieve high-quality dehazed renderings and reliable depth.
• The method offers significantly improved efficiency, enabling real-time dehazing and faster training compared to previous NeRF-based approaches, with potential applications in autonomous systems.

An Overview of "DehazeGS: Seeing Through Fog with 3D Gaussian Splatting"

The paper "DehazeGS: Seeing Through Fog with 3D Gaussian Splatting" introduces a novel approach to tackle the challenges of view synthesis in foggy scenes by employing a framework based on 3D Gaussian Splatting (3DGS). Existing methods for novel view synthesis, such as those leveraging Neural Radiance Fields (NeRF), often face limitations when handling scenes with scattering media due to high computational costs and difficulties in recovering fine details. DehazeGS addresses these issues by utilizing explicit Gaussian representations to model the interaction of light in foggy environments, facilitating more efficient and accurate 3D scene reconstruction and rendering.

Core Contributions

The research presents several key contributions to the field:

1. Novel Framework for Dehazing with 3DGS: DehazeGS is introduced as the first differentiable physics-based 3DGS dehazing model. It provides a mechanism to learn disentangled representations of participating media and clear scenes from only multi-view foggy images, outperforming existing methods in rendering quality and computational efficiency.
2. Explicit Gaussian Modeling of Scattering: The authors leverage 3D Gaussian Splatting to model the formation process of fog using a physically accurate forward rendering process. The framework employs a transmission function defined on each Gaussian to simulate light attenuation due to fog, distinguishing between the foggy layer and the clear scene.
3. Integration of Atmospheric Scattering Model: The methodology encompasses modeling atmospheric scattering based on a Gaussian approach. It blends clear Gaussian representations with learned transmission maps and atmospheric light parameters to synthesize foggy images.
4. Incorporation of Priors for Optimized Dehazing: DehazeGS incorporates prior information such as the Dark Channel Prior and Bright Channel Prior to improve the accuracy of transmission map estimation and enhance scene depth recovery. This leads to high-quality dehazed renderings with reliable depth estimations.
5. Performance and Efficiency: The proposed method achieves efficient real-time dehazing with training iterations substantially reduced compared to NeRF-based algorithms. The training process is optimized to deliver results with much shorter computational times without compromising rendering fidelity. Tests on both synthetic and real-world datasets validate its superior performance.

Theoretical and Practical Implications

From a theoretical perspective, this research enriches the understanding of how explicit representations like 3D Gaussians can be effectively utilized for complex media interactions typically encountered in foggy scenes. It offers a shift from the traditional NeRF-based approaches to a more explicit modeling paradigm, aligning with trends towards optimizing computational resources while maintaining high-quality outputs.

Practically, DehazeGS could significantly impact applications within autonomous driving, drone navigation, and robotic operations, where interpreting foggy environments accurately and swiftly is crucial. The model's efficiency and capacity to render dehazed images in less time makes it suitable for integration into real-time systems.

Speculations on Future Developments

Future developments might explore extending the framework to handle other atmospheric disturbances such as rain or snow, by adapting the Gaussian modeling process to different scattering characteristics. Furthermore, the integration of additional priors and constraints could potentially enhance the robustness and generalization of the model across vastly different scene structures and environmental conditions.

In summary, "DehazeGS: Seeing Through Fog with 3D Gaussian Splatting" presents an innovative method to improve the quality and efficiency of view synthesis in foggy environments. Through the explicit modeling of point cloud data using 3D Gaussians, it sets a precedent for future research aiming to solve similar challenges in complex visual environments.