Sub-pixel matching method for low-resolution thermal stereo images

Abstract: In the context of a localization and tracking application, we developed a stereo vision system based on cheap low-resolution 80x60 pixels thermal cameras. We proposed a threefold sub-pixel stereo matching framework (called ST for Subpixel Thermal): 1) robust features extraction method based on phase congruency, 2) rough matching of these features in pixel precision, and 3) refined matching in sub-pixel accuracy based on local phase coherence. We performed experiments on our very low-resolution thermal images (acquired using a stereo system we manufactured) as for high-resolution images from a benchmark dataset. Even if phase congruency computation time is high, it was able to extract two times more features than state-of-the-art methods such as ORB or SURF. We proposed a modified version of the phase correlation applied in the phase congruency feature space for sub-pixel matching. Using simulated stereo, we investigated how the phase congruency threshold and the sub-image size of sub-pixel matching can influence the accuracy. We then proved that given our stereo setup and the resolution of our images, being wrong of 1 pixel leads to a 500 mm error in the Z position of the point. Finally, we showed that our method could extract four times more matches than a baseline method ORB + OpenCV KNN matching on low-resolution images. Moreover, our matches were more robust. More precisely, when projecting points of a standing person, ST got a standard deviation of 300 mm when ORB + OpenCV KNN gave more than 1000 mm.