Feedback Network for Image Super-Resolution

Abstract: Recent advances in image super-resolution (SR) explored the power of deep learning to achieve a better reconstruction performance. However, the feedback mechanism, which commonly exists in human visual system, has not been fully exploited in existing deep learning based image SR methods. In this paper, we propose an image super-resolution feedback network (SRFBN) to refine low-level representations with high-level information. Specifically, we use hidden states in an RNN with constraints to achieve such feedback manner. A feedback block is designed to handle the feedback connections and to generate powerful high-level representations. The proposed SRFBN comes with a strong early reconstruction ability and can create the final high-resolution image step by step. In addition, we introduce a curriculum learning strategy to make the network well suitable for more complicated tasks, where the low-resolution images are corrupted by multiple types of degradation. Extensive experimental results demonstrate the superiority of the proposed SRFBN in comparison with the state-of-the-art methods. Code is avaliable at https://github.com/Paper99/SRFBN_CVPR19.

• The paper introduces a novel feedback mechanism in SRFBN that uses recurrent loops to iteratively refine low-level features and boost reconstruction quality.
• The feedback block design combines up- and down-sampling layers with dense skip connections, enabling efficient feature enrichment with fewer parameters.
• A curriculum learning strategy progressively challenges the network with complex degradation models, resulting in enhanced performance across standard datasets.

Feedback Network for Image Super-Resolution: An Expert Overview

The paper introduces the Super-Resolution Feedback Network (SRFBN), a novel approach in the field of image super-resolution (SR) that utilizes a feedback mechanism to enhance the reconstruction quality of high-resolution (HR) images from low-resolution (LR) counterparts. The proposed SRFBN aims to refine low-level features by integrating high-level representations through recurrent feedback loops, a method that draws inspiration from feedback mechanisms found in human cognition.

Core Contributions

1. Feedback Mechanism: The SRFBN integrates a feedback mechanism, which allows high-level information to be reintroduced into the network to correct earlier states. This is implemented using recurrent neural network (RNN) structures and provides strong early reconstruction abilities with fewer parameters.
2. Feedback Block Design: The feedback block (FB) within SRFBN handles feedback loops and enhances representation via up- and down-sampling layers and dense skip connections. This design parallelizes operations efficiently, thereby enriching high-level feature representations.
3. Curriculum Learning Strategy: The paper introduces a curriculum learning strategy for complex degradation models. By adjusting the learning targets from easy to difficult, the network progressively learns to reconstruct more challenging degraded images effectively.

Experimental Evaluation

Extensive experiments showcase the SRFBN's superior performance over state-of-the-art methods. The network was validated on standard datasets such as Set5, Set14, B100, Urban100, and Manga109, demonstrating quantitative and qualitative improvements. The approach's robustness was further tested under various degradation models, including bicubic downsampling (BI), Gaussian-blur and downsampling (BD), and bicubic downsampling with added Gaussian noise (DN).

Notable Numerical Results

• Set5 Dataset: The SRFBN achieved a PSNR of 32.11 dB for a scale factor of ×4, highlighting its potent early-stage image reconstruction capabilities.
• Urban100 Dataset: The model showed pronounced improvements, evidencing strong preservation and enhancement of structural details in urban scene images.

Theoretical and Practical Implications

The SRFBN extends the architectural possibilities for SR networks by incorporating feedback loops, breaking away from traditional feedforward designs. This approach not only offers a different pathway for model optimization and learning efficiency but also sheds light on enhancing generalization by iterative refinement.

Future Directions

The concept of feedback in SR networks opens several avenues for future exploration. Possible improvements include expanding to multi-scale feedback networks, integrating advanced forms of gating mechanisms, or combining with other state-of-the-art architectures for diversified input adaptations.

In conclusion, the SRFBN presents a significant stride in image super-resolution technologies, combining innovative architectural design with a thoughtful learning strategy, ultimately leading to enhanced visual fidelity with computational efficiency. The exploration of feedback mechanisms in deep learning models hints at broader applications beyond SR, potentially benefiting various domains within computer vision.