Institutional Repository of Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
|Thesis Advisor||潘春洪 ; 孟高峰|
|Place of Conferral||中国科学院自动化研究所|
|Keyword||图像质量评价 视觉感知质量 无参考图像质量评价方法 深度学习 向量回归 目标导向池化 视觉注意力 可学习池化 强化递归排序|
1. 提出了一种基于向量回归（vector regression）与目标导向池化（object oriented pooling）的无参考图像质量评价方法。该方法探究了质量评价中因个体差异而存在的不确定性，并据此设计了一组“置信”分数以隐式地度量图像质量隶属于相应评分区间的概率。向量回归框架的核心思想是用“置信”分数向量表征图像视觉质量。与现有方法直接学习从图像到平均意见分数的一元映射不同，基于向量回归的图像质量评价模型学习从图像到“置信”分数向量的多元映射。另外，目标导向池化的核心思想是在池化阶段引入语义信息。该方法赋予图像中类物体区域（object-like region）更大的权重，使得池化操作更符合视觉特性从而进一步提升模型质量评价性能。多个数据库上的大量实验结果证实了所提方法对失真图像质量评价的有效性。
2. 提出了一种基于可学习注意力池化（learnable attention-based pooling）的无参考图像质量评价方法。该方法将可学习池化引入到无参考图像质量评价中，其核心是以数据驱动的方式学得既有效又符合视觉注意力的池化方法。具体地，通过内置可学习注意力模块提出了注意力池化网络。该网络由两个分支构成，分别用于学习评价图像局部质量和分配池化阶段注意力权重。为辅助模型更好地训练，在训练阶段进一步为注意力池化网络的两个分支引入相关约束。对于受视觉关注的图像区域，该约束将对局部质量分数与全局质量分数相背离的情形进行惩罚。在实验层面，大量实验结果充分证实了所提无参考评价方法优异的性能，另外所学池化方法的有效性和可解释性也得到了验证。
Nowadays digital images are everywhere. Unfortunately, the images are often distorted at various stages of their life cycle, e.g., during acquisition, subsequent compression and transmission. The introduced distortions may probably result in a degradation of visual quality. Image quality assessment (IQA) aims to quantitatively measure the degradation and predict the perceived quality. An effective and efficient evaluation on image quality plays an important role in many practical applications and is crucial to guarantee high quality of experience (QoE) for human viewers. In recent years, there has been a growing interest in developing objective IQA algorithms for an automated and real-time quality evaluation. Thereinto, no-reference image quality assessment (NR-IQA) refers to the technique of directly predicting the visual quality of distorted images without accessing any information from reference images. Since the requirement of reference is often problematic in practice, research on developing effective NR-IQA models is of great significance.
This dissertation focuses on NR-IQA via deep learning. Several effective methods are developed by incorporating the uncertainty in quality assessment, visual attention mechanism, learnable pooling, reinforcement learning and list-wise learning to rank. Majority of these techniques are the first time proposed in this research field. The major contributions are summarized as follows.
1. An effective NR-IQA method based on vector regression and object oriented pooling is proposed. In this work, the uncertainty factors in quality assessment due to individual differences are well explored, and accordingly a vector of belief scores is developed to implicitly measure the probabilities of an image belonging to the corresponding quality intervals. The core idea behind the vector regression framework is to characterize image quality with the introduced belief scores. Unlike previous ones that map the extracted features directly to a quality score, the presented vector regression based NR-IQA model yields a vector of belief scores for the input image. Moreover, an object oriented pooling strategy is proposed to further improve the performance by incorporating semantic information of image contents. According to this strategy, regions occupied by objects will be assigned more weights in the pooling phase, leading to a more accurate quality assessment. Extensive experiments on benchmark databases demonstrate that our approach achieves state-of-the-art performance and shows a great generalization ability.
2. An NR-IQA method via learnable attention-based pooling is proposed. This work focuses on developing an effective learnable pooling. The core is to achieve a pooling learned in a data-driven manner which not only can benefit the IQA performance but also correlates well with visual attention. Specifically, an attention-based pooling network is developed by incorporating a built-in attention module. This network allows for a joint learning of local quality evaluation and attention weight assignment. A correlation constraint between the estimated local quality and attention weight in the network is further introduced to regulate the training. The constraint penalizes the case in which the local quality estimation on a region attracting more attention differs a lot from the overall quality score. Comprehensive experimental results show that the proposed method achieves state-of-the-art prediction accuracy. Moreover, the effectiveness and interpretability of the learned attention-based pooling model are also verified.
3. An effective NR-IQA method based on reinforcement recursive list-wise ranking is proposed. The presented method is opinion-unaware, which does not require images with subjective scores for training. In specific, the NR-IQA is formulated as a recursive list-wise ranking problem, which learns to rank a list of images with implicit quality measures. The use of image lists as learning instances allows our method to optimize the overall quality ordering directly. During training, the recursive ranking process can be modeled as a Markov decision process. Policy-based reinforcement learning is adopted to train the IQA model, in which no ground-truth quality scores or ranking lists are necessary for learning. Experimental results demonstrate the superior performance of our approach in comparison with existing opinion-unaware NR-IQA methods. Furthermore, our approach can compete with the most effective opinion-aware methods even without subjective score supervision.
|顾杰. 基于深度学习的无参考图像质量评价方法研究[D]. 中国科学院自动化研究所. 中国科学院大学,2019.|
|Files in This Item:|
|博士论文.pdf（12490KB）||学位论文||开放获取||CC BY-NC-SA||Application Full Text|
|Recommend this item|
|Export to Endnote|
|Similar articles in Google Scholar|
|Similar articles in Baidu academic|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.