A Systematic Survey of Deep Learning-based Single-Image Super-Resolution

Abstract: Single-image super-resolution (SISR) is an important task in image processing, which aims to enhance the resolution of imaging systems. Recently, SISR has made a huge leap and has achieved promising results with the help of deep learning (DL). In this survey, we give an overview of DL-based SISR methods and group them according to their design targets. Specifically, we first introduce the problem definition, research background, and the significance of SISR. Secondly, we introduce some related works, including benchmark datasets, upsampling methods, optimization objectives, and image quality assessment methods. Thirdly, we provide a detailed investigation of SISR and give some domain-specific applications of it. Fourthly, we present the reconstruction results of some classic SISR methods to intuitively know their performance. Finally, we discuss some issues that still exist in SISR and summarize some new trends and future directions. This is an exhaustive survey of SISR, which can help researchers better understand SISR and inspire more exciting research in this field. An investigation project for SISR is provided at https://github.com/CV-JunchengLi/SISR-Survey.

• The paper outlines a systematic classification of DL-based SISR methods into simulation, real-world, and domain-specific categories.
• It emphasizes advanced techniques such as residual learning, attention mechanisms, and transformer models to achieve state-of-the-art reconstruction.
• The study identifies practical challenges like degradation modeling and proposes future research directions for lightweight and adaptive SISR solutions.

A Systematic Survey of Deep Learning-based Single-Image Super-Resolution

This paper presents a comprehensive survey of deep learning (DL)-based methods for single-image super-resolution (SISR), a critical task aimed at enhancing the resolution of images. In recent years, aided by the advancements in deep learning techniques, SISR has progressed significantly, achieving remarkable results in a variety of applications including security and surveillance, medical imaging, and video enhancement. This survey methodically categorizes existing DL-based SISR approaches by their design targets, providing a structured understanding of the field, and identifies current challenges and future research directions.

The paper begins by defining the SISR problem and discusses its significance. SISR aims to reconstruct high-resolution (HR) images from their degraded low-resolution (LR) counterparts. The authors provide an overview of traditional methods, highlighting the ill-posed nature of SISR and the advantages of DL-based methods over conventional interpolation and sparse coding techniques. DL approaches, leveraging convolutional neural networks (CNNs), offer an end-to-end framework capable of achieving state-of-the-art (SOTA) performance by effectively learning the complex relationships between LR and HR images.

A key contribution of this survey is its classification of DL-based SISR methods into three main categories: Simulation SISR, Real-World SISR, and Domain-Specific Applications. Each category is further subdivided based on specific design objectives.

1. Simulation SISR Methods: These are predominantly focused on achieving high reconstruction accuracy on synthetically degraded data. The paper reviews methodologies under subcategories such as efficient network/mechanism design methods, perceptual quality methods, and additional information utilization methods. Techniques like residual learning, attention mechanisms, multi-scale learning, and the integration of generative adversarial networks (GANs) for enhanced perceptual quality are explored in detail.
2. Real-World SISR Methods: The survey identifies the challenges of applying SISR in real-world scenarios, where degradation models are complex and unknown. The paper discusses explicit and implicit degradation modeling methods. Explicit methods estimate degradation parameters such as blur kernels and noise, while implicit approaches employ generative models to learn and adapt to real degradation processes. The authors also touch upon scale-arbitrary SISR techniques, which address the limitation of fixed upscale factors inherent in many existing models.
3. Domain-Specific Applications: SISR techniques tailored for specific applications are reviewed, including stereo image SR, remote sensing SR, light field SR, face image SR, hyperspectral SR, and medical image SR. These domain-specific adaptations leverage unique data characteristics to improve performance, highlighting the flexibility and applicability of SISR across diverse fields.

The survey provides numerical results for various SISR models across standard benchmark datasets, allowing for a comparative analysis of these methods. Notably, the inclusion of recent transformer-based architectures and diffusion model-based methods showcases the progressive shift towards models capable of capturing long-range dependencies and delivering enhanced visual fidelity.

The paper concludes by discussing open challenges and future directions for SISR research. These include developing lightweight models for deployment on edge devices, creating flexible architectures capable of adapting to variable scaling factors and degradations, and exploring novel loss functions and evaluation metrics that align better with human perceptual judgment. Furthermore, the integration of SISR with high-level computer vision tasks and the design of efficient real-world SISR strategies are emphasized as burgeoning areas of research.

Overall, this survey serves as a valuable resource for researchers in the field of image processing, providing insights into both the current landscape and future trajectory of DL-based SISR. It encourages the exploration of innovative solutions to overcome existing limitations and advance the practical applicability of SISR technologies.