Multispectral image fusion based on super pixel segmentation

Abstract: Multispectral image fusion is a computer vision process that is essential to remote sensing. For applications such as dehazing and object detection, there is a need to offer solutions that can perform in real-time on any type of scene. Unfortunately, current state-of-the-art approaches do not meet these criteria as they need to be trained on domain-specific data and have high computational complexity. This paper focuses on the task of fusing color (RGB) and near-infrared (NIR) images as this the typical RGBT sensors, as in multispectral cameras for detection, fusion, and dehazing. Indeed, the NIR channel has the ability to capture details not visible in RGB and see beyond haze, fog, and clouds. To combine this information, a novel approach based on superpixel segmentation is designed so that multispectral image fusion is performed according to the specific local content of the images to be fused. Therefore, the proposed method produces a fusion that contains the most relevant content of each spectrum. The experiments reported in this manuscript show that the novel approach better preserve details than alternative fusion methods.