Manifold-Inspired Single Image Interpolation

Abstract: Manifold models consider natural-image patches to be on a low-dimensional manifold embedded in a high dimensional state space and each patch and its similar patches to approximately lie on a linear affine subspace. Manifold models are closely related to semi-local similarity, a well-known property of natural images, referring to that for most natural-image patches, several similar patches can be found in its spatial neighborhood. Many approaches to single image interpolation use manifold models to exploit semi-local similarity by two mutually exclusive parts: i) searching each target patch's similar patches and ii) operating on the searched similar patches, the target patch and the measured input pixels to estimate the target patch. Unfortunately, aliasing in the input image makes it challenging for both parts. A very few works explicitly deal with those challenges and only ad-hoc solutions are proposed. To overcome the challenge in the first part, we propose a carefully-designed adaptive technique to remove aliasing in severely aliased regions, which cannot be removed from traditional techniques. This technique enables reliable identification of similar patches even in the presence of strong aliasing. To overcome the challenge in the second part, we propose to use the aliasing-removed image to guide the initialization of the interpolated image and develop a progressive scheme to refine the interpolated image based on manifold models. Experimental results demonstrate that our approach reconstructs edges with both smoothness along contours and sharpness across profiles, and achieves an average Peak Signal-to-Noise Ratio (PSNR) significantly higher than existing model-based approaches.