On the Use of Deep Learning for Blind Image Quality Assessment

Abstract: In this work we investigate the use of deep learning for distortion-generic blind image quality assessment. We report on different design choices, ranging from the use of features extracted from pre-trained Convolutional Neural Networks (CNNs) as a generic image description, to the use of features extracted from a CNN fine-tuned for the image quality task. Our best proposal, named DeepBIQ, estimates the image quality by average pooling the scores predicted on multiple sub-regions of the original image. The score of each sub-region is computed using a Support Vector Regression (SVR) machine taking as input features extracted using a CNN fine-tuned for category-based image quality assessment. Experimental results on the LIVE In the Wild Image Quality Challenge Database and on the LIVE Image Quality Assessment Database show that DeepBIQ outperforms the state-of-the-art methods compared, having a Linear Correlation Coefficient (LCC) with human subjective scores of almost 0.91 and 0.98 respectively. Furthermore, in most of the cases, the quality score predictions of DeepBIQ are closer to the average observer than those of a generic human observer.

• The paper introduces DeepBIQ, a novel approach leveraging transfer learning and fine-tuning of CNNs for blind image quality assessment.
• It employs feature pooling from image sub-regions to effectively capture diverse distortions like JPEG compression, noise, and blur.
• Experiments on LIVE and synthetic databases yield an LCC of ~0.91, significantly improving on previous state-of-the-art performance.

An Analysis of Deep Learning Approaches for Blind Image Quality Assessment

The paper, authored by Simone Bianco, Luigi Celona, Paolo Napoletano, and Raimondo Schettini, explores the application of deep learning methodologies specifically tailored to the problem of Blind Image Quality Assessment (BIQA). The investigation examines multiple facets of deep learning models, including the use of pre-trained Convolutional Neural Networks (CNNs), fine-tuning strategies, and features extracted from CNN models for predictive purposes in distortion-generic NR-IQA tasks.

The paper introduces a methodology termed DeepBIQ, which relies on CNNs to capture and model non-distorted image statistics that represent visual quality. The research utilizes various CNN architectures pre-trained on tasks such as ImageNet and Places for image recognition to explore their efficacy in capturing image artifacts, including JPEG compression, noise, and blur.

Three main components form the core of the study: the use of pre-trained CNNs with transfer learning applied to BIQA, feature and prediction pooling strategies to gather comprehensive image quality data from multiple image sub-regions, and the customization (fine-tuning) of CNNs for NR-IQA tasks to improve model performance. These approaches are designed to address the challenges of evaluating complex distortions in real-world photographic images.

The authors conducted rigorous experiments using the LIVE In the Wild Image Quality Challenge Database and subsequently evaluated their methodology on several legacy synthetically distorted image quality databases, including LIVE, CSIQ, TID2008, and TID2013. The DeepBIQ approach achieves a remarkable Linear Correlation Coefficient (LCC) of approximately 0.91 in the tested scenarios, indicating a high agreement with human subjective scores and improving on the previous state-of-the-art by a significant margin.

The paper highlights the potential of leveraging large, diverse datasets for initial CNN training to obtain generalized features capable of coping with multiple distortion types. The research's findings demonstrate that image quality assessment, a traditionally challenging task given the subjective nature of human perceptual evaluation, can be effectively modeled with deep learning techniques by employing data-driven strategies and fine-tuning procedures.

The practical implications of this research are substantial. DeepBIQ demonstrates advancement in predictive accuracy for real-world images subjected to unknown and multiple distortions, suggesting applications in optimizing media content distribution and enhancing digital image processing workflows. The potential for further exploration includes the integration of advanced neural architectures or hybrid models to extend and refine these capabilities.

In summary, this paper delineates a clear path from traditional subjective methods of image quality assessment to modern data-driven, automated approaches facilitated by deep learning. The work not only provides empirical evidence for deep learning's application to BIQA but also underscores a framework for future research incorporating evolving neural network models and more expansive datasets.