Environment reconstruction on depth images using Generative Adversarial Networks

Abstract: Robust perception systems are essential for autonomous vehicle safety. To navigate in a complex urban environment, it is necessary precise sensors with reliable data. The task of understanding the surroundings is hard by itself; for intelligent vehicles, it is even more critical due to the high speed in which the vehicle navigates. To successfully navigate in an urban environment, the perception system must quickly receive, process, and execute an action to guarantee both passenger and pedestrian safety. Stereo cameras collect environment information at many levels, e.g., depth, color, texture, shape, which guarantee ample knowledge about the surroundings. Even so, when compared to human, computational methods lack the ability to deal with missing information, i.e., occlusions. For many perception tasks, this lack of data can be a hindrance due to the environment incomplete information. In this paper, we address this problem and discuss recent methods to deal with occluded areas inference. We then introduce a loss function focused on disparity and environment depth data reconstruction, and a Generative Adversarial Network (GAN) architecture able to deal with occluded information inference. Our results present a coherent reconstruction on depth maps, estimating regions occluded by different obstacles. Our final contribution is a loss function focused on disparity data and a GAN able to extract depth features and estimate depth data by inpainting disparity images.