CASIA OpenIR  > 模式识别国家重点实验室  > 先进数据分析与学习
基于深度学习的无参考图像质量评价方法研究
顾杰
Subtype博士
Thesis Advisor潘春洪 ; 孟高峰
2019-05
Degree Grantor中国科学院大学
Place of Conferral中国科学院自动化研究所
Degree Discipline模式识别与智能系统
Keyword图像质量评价 视觉感知质量 无参考图像质量评价方法 深度学习 向量回归 目标导向池化 视觉注意力 可学习池化 强化递归排序
Abstract

数字图像在其采集、压缩及传输等过程中通常会遭受不同程度的失真降质。图像质量评价旨在度量失真对图像视觉感知质量的影响。高效准确的图像质量评价在诸多实际应用中均发挥着关键作用,是保证用户视觉感知体验的重要基础。近年来,客观图像质量评价方法因其可实现自动高效的质量预测获得了研究人员的广泛关注。其中,无参考客观方法能够在参考图像信息不可用的情况下直接对失真图像进行质量评价。由于在大多数实际应用场景中参考图像无法采集或获取代价较大,因此开发有效的无参考图像质量评价方法具有十分重要的研究意义和应用价值。
在图像质量评价中,受人类主观因素影响,人们给出的质量分数往往具有较大的随机性。同时,图像质量的评价与人类视觉感知特性密切相关。基于这些观察和认识,本文创新性地将向量回归、视觉注意力机制、可学习池化以及强化排序等思想引入深度学习框架,先后提出了多种有效的无参考图像质量评价方法。论文的主要贡献包括以下几个方面:

1. 提出了一种基于向量回归(vector regression)与目标导向池化(object oriented pooling)的无参考图像质量评价方法。该方法探究了质量评价中因个体差异而存在的不确定性,并据此设计了一组“置信”分数以隐式地度量图像质量隶属于相应评分区间的概率。向量回归框架的核心思想是用“置信”分数向量表征图像视觉质量。与现有方法直接学习从图像到平均意见分数的一元映射不同,基于向量回归的图像质量评价模型学习从图像到“置信”分数向量的多元映射。另外,目标导向池化的核心思想是在池化阶段引入语义信息。该方法赋予图像中类物体区域(object-like region)更大的权重,使得池化操作更符合视觉特性从而进一步提升模型质量评价性能。多个数据库上的大量实验结果证实了所提方法对失真图像质量评价的有效性。

2. 提出了一种基于可学习注意力池化(learnable attention-based pooling)的无参考图像质量评价方法。该方法将可学习池化引入到无参考图像质量评价中,其核心是以数据驱动的方式学得既有效又符合视觉注意力的池化方法。具体地,通过内置可学习注意力模块提出了注意力池化网络。该网络由两个分支构成,分别用于学习评价图像局部质量和分配池化阶段注意力权重。为辅助模型更好地训练,在训练阶段进一步为注意力池化网络的两个分支引入相关约束。对于受视觉关注的图像区域,该约束将对局部质量分数与全局质量分数相背离的情形进行惩罚。在实验层面,大量实验结果充分证实了所提无参考评价方法优异的性能,另外所学池化方法的有效性和可解释性也得到了验证。

3. 提出了一种无需主观评分训练的“意见无关”(opinion-unaware)无参考图像质量评价方法。该方法将深度强化学习和序列排序学习引入到图像质量评价中。核心思想是将无参考图像质量评价问题形式化为一个递归序列排序问题,并采用强化学习方法训练模型。具体地,该方法使用图像序列作为训练样本,学习如何对序列中图像按其视觉质量进行排序。在训练阶段,递归排序过程可建模为马尔可夫决策过程。相应地图像质量评价模型能用基于策略的强化学习方法进行灵活有效地逐步训练。实验结果表明所提方法能够准确预测失真图像的视觉质量,其性能不仅胜过当前效果最好的“意见无关”无参考评价方法,甚至优于许多新近提出的需用主观质量分数进行监督训练的无参考方法。

Other Abstract

Nowadays digital images are everywhere. Unfortunately, the images are often distorted at various stages of their life cycle, e.g., during acquisition, subsequent compression and transmission. The introduced distortions may probably result in a degradation of visual quality. Image quality assessment (IQA) aims to quantitatively measure the degradation and predict the perceived quality. An effective and efficient evaluation on image quality plays an important role in many practical applications and is crucial to guarantee high quality of experience (QoE) for human viewers. In recent years, there has been a growing interest in developing objective IQA algorithms for an automated and real-time quality evaluation. Thereinto, no-reference image quality assessment (NR-IQA) refers to the technique of directly predicting the visual quality of distorted images without accessing any information from reference images. Since the requirement of reference is often problematic in practice, research on developing effective NR-IQA models is of great significance.

This dissertation focuses on NR-IQA via deep learning. Several effective methods are developed by incorporating the uncertainty in quality assessment, visual attention mechanism, learnable pooling, reinforcement learning and list-wise learning to rank. Majority of these techniques are the first time proposed in this research field. The major contributions are summarized as follows.

1. An effective NR-IQA method based on vector regression and object oriented pooling is proposed. In this work, the uncertainty factors in quality assessment due to individual differences are well explored, and accordingly a vector of belief scores is developed to implicitly measure the probabilities of an image belonging to the corresponding quality intervals. The core idea behind the vector regression framework is to characterize image quality with the introduced belief scores. Unlike previous ones that map the extracted features directly to a quality score, the presented vector regression based NR-IQA model yields a vector of belief scores for the input image. Moreover, an object oriented pooling strategy is proposed to further improve the performance by incorporating semantic information of image contents. According to this strategy, regions occupied by objects will be assigned more weights in the pooling phase, leading to a more accurate quality assessment. Extensive experiments on benchmark databases demonstrate that our approach achieves state-of-the-art performance and shows a great generalization ability.

2. An NR-IQA method via learnable attention-based pooling is proposed. This work focuses on developing an effective learnable pooling. The core is to achieve a pooling learned in a data-driven manner which not only can benefit the IQA performance but also correlates well with visual attention. Specifically, an attention-based pooling network is developed by incorporating a built-in attention module. This network allows for a joint learning of local quality evaluation and attention weight assignment. A correlation constraint between the estimated local quality and attention weight in the network is further introduced to regulate the training. The constraint penalizes the case in which the local quality estimation on a region attracting more attention differs a lot from the overall quality score. Comprehensive experimental results show that the proposed method achieves state-of-the-art prediction accuracy. Moreover, the effectiveness and interpretability of the learned attention-based pooling model are also verified.

3. An effective NR-IQA method based on reinforcement recursive list-wise ranking is proposed. The presented method is opinion-unaware, which does not require images with subjective scores for training. In specific, the NR-IQA is formulated as a recursive list-wise ranking problem, which learns to rank a list of images with implicit quality measures. The use of image lists as learning instances allows our method to optimize the overall quality ordering directly. During training, the recursive ranking process can be modeled as a Markov decision process. Policy-based reinforcement learning is adopted to train the IQA model, in which no ground-truth quality scores or ranking lists are necessary for learning. Experimental results demonstrate the superior performance of our approach in comparison with existing opinion-unaware NR-IQA methods. Furthermore, our approach can compete with the most effective opinion-aware methods even without subjective score supervision.

Pages118
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/23857
Collection模式识别国家重点实验室_先进数据分析与学习
Recommended Citation
GB/T 7714
顾杰. 基于深度学习的无参考图像质量评价方法研究[D]. 中国科学院自动化研究所. 中国科学院大学,2019.
Files in This Item:
File Name/Size DocType Version Access License
博士论文.pdf(12490KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[顾杰]'s Articles
Baidu academic
Similar articles in Baidu academic
[顾杰]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[顾杰]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.