毕业生知识库

图像作为重要的信息载体，其质量会直接影响信息传递效果。在实际情况中，影响图像质量的因素主要来自图像采集、成像和传输过程。这些过程使原始高频细节丰富的高分辨率图像可能受到模糊、噪声、下采样、压缩等退化影响，成为视觉效果较差的低分辨率图像。图像超分辨率作为底层的数字图像处理任务，旨在从低分辨率图像中恢复出退化过程损失的高频信息，重建出对应的高分辨率图像。这对于改善图像的视觉效果和提升高级视觉任务（如识别、检测和分割等）的性能都具有重要的研究和应用价值。

根据低分辨率图像包含的退化方式是否已知，图像超分辨率主要分为退化方式已知(即非盲超分辨率)和退化方式未知(即盲超分辨率)两大场景。尽管基于深度学习的图像超分辨率算法研究已经取得了显著进展，但实际应用中仍存在很多挑战。非盲超分辨率场景下的关注点是超分辨率模型设计，需要解决模型计算成本与超分辨率效果之间的平衡性问题、图像退化与超分辨率过程的匹配性问题等。盲超分辨率场景下的重点问题则是退化估计，良好的退化估计效果可以缓解训练数据与测试数据的数据域差异，是保证超分辨率模型对不同退化方式具有鲁棒性的重要前提。因此，本文以低分辨图像包含的退化方式为切入点，根据退化方式从已知到未知，退化类型从单一到多样的顺序，对上述多种退化场景下的图像超分辨率问题进行研究。本文的主要内容和创新点归纳如下：

1. 基于双重特征融合的轻量化图像非盲超分辨率算法。该算法主要面向低分辨率图像的退化方式已知且特定的场景。大部分基于深度学习的图像非盲超分辨率算法倾向于不断增大网络的参数量和计算复杂度，以更好地学习低分辨率图像与高分辨率图像之间的非线性映射关系，但这导致模型内存占用大、推理过程耗能高，不适用于移动设备等实际应用任务。为了缓解该问题，本文提出基于双重特征融合的图像非盲超分辨率算法，利用分组卷积设计轻量化的基础网络结构，同时重点从“特征复用”的角度出发，提出双重特征融合策略，主要包括局部特征融合模块和全局特征融合模块。首先，局部特征融合模块利用特征平衡连接方式，对不同深度的特征按照通道维度进行自适应地局部融合。进一步地，全局特征融合模块利用空间注意力机制对局部融合后的特征再次进行迭代式地融合。实验表明，该算法可以提高特征利用率，更好地平衡模型的超分辨率效果和计算成本，在多个数据集上均有显著效果。

2. 基于可逆流模型的图像退化-非盲超分辨率算法}。该算法主要面向低分辨率图像的退化方式已知且非特定(可自行设计)的场景。在图像非盲超分辨率研究中，低分辨率图像的退化方式大多是预定义的，与超分辨率方法相互独立，但这可能导致二者匹配性较差，影响超分辨率效果的上限。针对该问题，本文自主设计了与图像超分辨率匹配性更高的退化方式，提出一种基于可逆流模型的图像退化-非盲超分辨率算法。该算法基于编码-解码网络建模图像退化过程及超分辨率过程，实现二者的联合优化，提高匹配程度。并且进一步提出基于流模型的可逆转换模块，实现对于“超分辨率最优”的退化特征与具有良好视觉质量的低分辨率图像之间的相互转换，从而缓解显式退化监督对于后续超分辨率效果的限制，在保证退化后的低分辨率图像具有良好视觉效果的同时，显著提高超分辨率重建的效果。

3. 基于在线退化估计的图像盲超分辨率算法。该算法主要面向低分辨率图像的退化方式未知且退化类型主要为“模糊”的场景。当前大部分图像盲超分辨率算法使用预定义退化方式合成的数据进行离线训练，忽略了合成训练数据与待测试图像的数据域差异。此外，离线训练后的模型参数固定，不能针对不同的退化方式(如不同形状的高斯模糊核)进行灵活调整。针对这些问题，本文提出一种基于在线退化估计的图像盲超分辨率算法。结合最大后验概率，该算法设计了内部学习支路和外部学习支路。内部学习支路可以无监督地在线估计低分辨率测试图像的退化方式，外部学习支路则可以根据估计出的退化方式利用外部高分辨率图像合成伪样本对，训练超分辨率模型。两支路交替优化，进而自适应地学习测试图像包含的退化方式，缩小训练图像与测试图像之间的数据域差异。大量实验表明，该算法能够对不同的模糊核进行较好的估计，针对不同低分辨率图像的退化方式训练特定的超分辨率模型，取得更加鲁棒的超分辨率效果。

4. 基于不确定性退化估计的图像盲超分辨率算法。该算法主要面向低分辨率图像的退化方式未知且退化类型多样(同时包含模糊、噪声、压缩等退化类型)的场景。在图像盲超分辨率研究中，大部分退化估计算法只能估计特定的退化方式，或者生成具有特定退化方式的低分辨率训练样本。但准确估计退化方式难度很大，估计偏差可能导致合成的训练样本和低分辨率测试图像之间存在数据域差异，在测试图像同时包含多种退化类型时，估计偏差可能进一步增大，进而对超分辨率模型的效果产生不利影响。针对该问题，本文结合不确定性学习，将退化估计建模为退化参数回归问题，提出一种基于不确定性学习的退化估计算法。该算法将退化参数估计为特定的分布，而非一个特定的数值。这使得真实的退化方式更有可能被包含在估计的退化空间中，缓解退化估计偏差，缩小测试图像与合成训练样本之间的数据域差异，进而提高超分辨率效果。此外，退化参数回归模型适用于多种退化类型的估计任务，可同时对模糊、噪声、压缩等退化类型的退化参数进行估计。该方法的有效性在合成数据和真实数据上均得到验证。

Image is an important medium for information transmission and the representation performance is directly determined by image quality. Image quality can be adversely affected during the process of acquisition, imaging, and transmission. During these processes, the original high-resolution image may suffer random or artificial degradations such as blurring, noise, downsampling, and compression, which consequently results in a low-resolution image with poor visual quality. Image super-resolution, a low-level vision task, aims to recover high-frequency information lost during the degradation process and reconstruct a high-resolution image from the low-resolution counterpart, which is significant for image quality enhancement and high-level vision tasks such as classification, detection, and segmentation.

There are two main directions in image super-resolution, that is, non-blind image super-resolution with predefined degradations and blind image super-resolution with unknown degradations. Nowadays,  deep learning-based image super-resolution has made remarkable progress, but there are still many challenges in different degradation scenarios. For instance, in non-blind image super-resolution, the main problems are the trade-off between super-resolution performance and computation cost, and the compatibility between the degradation process and the super-resolution process. As for blind image super-resolution, how to estimate unknown degradations and how to make the super-resolution model more robust for different degradations are the main challenges. This thesis focuses on how to improve image super-resolution performance in these challenging degradation scenarios. Specifically, we study the key issues of image super-resolution from simple and known degradation scenarios to complex and unknown scenarios. The major contributions of this thesis are summarized as follows:

1. A dual feature aggregation network for lightweight non-blind image super-resolution.  In scenarios where the degradations of low-resolution images are known and fixed, most deep learning-based super-resolution models usually have a large model size and high computational complexity, which hinders the application in devices with limited memory and computing power. Some lightweight super-resolution methods solve this issue by directly designing shallower architectures, but it will adversely affect the representation capability of convolutional neural networks. To address this issue, we propose the dual feature aggregation strategy for lightweight image super-resolution. It enhances the feature representation via feature reuse, which only introduces marginal computational cost. Thus, a smaller model could achieve better cost-effectiveness with the dual feature aggregation strategy. Specifically, it consists of a local aggregation module and a global aggregation module, which cooperate to further aggregate hierarchical features adaptively along the channel and spatial dimensions. In addition, we propose a compact basic building block via group convolution to extract hierarchical features in a more efficient way. Extensive experiments suggest that the proposed network performs favorably against state-of-the-art methods in terms of visual quality, memory footprint, and computational complexity.

2. A joint image degradation and non-blind super-resolution network via invertible flow guidance. This algorithm is mainly aimed at scenarios where the degradation process of low-resolution images is known and not specific (self-designed). In non-blind super-resolution, the degradation process is usually predefined and independent of the super-resolution process. However, this may make the degradation process incompatible with the subsequent super-resolution process and limit the upper bound of super-resolution performance. Towards this issue, we design a degradation model which can cooperate better with the super-resolution model. Specifically, we model image degradation and super-resolution as a joint encoding-decoding task. A flow guidance module is further proposed to transform the downscaled feature to a visually plausible image during degradation and transform it back during super-resolution. Benefitting from its invertibility, the downscaled feature could get rid of explicit degradation supervision and become “super-resolution optimal”. The flow guidance module allows us to remove the restrictions on the degradation module and optimize the degradation and super-resolution modules in an end-to-end manner. In this way, these two modules could cooperate to maximize super-resolution performance. Extensive experiments demonstrate that the proposed method can achieve state-of-the-art performance on both downscaled and super-resolved images.

3. An online degradation estimation method for blind image super-resolution. The algorithm is mainly aimed at scenarios where the degradations of low-resolution images are unknown and the main degradation type is "blurry". In such scenarios, most deep learning-based super-resolution methods are trained on samples synthesized by predefined degradations (e.g. bicubic downsampling), regardless of the domain gap between training and testing data. During testing, they super-resolve all images by the same set of model weights, ignoring the degradation variety. As a result, these methods may suffer a performance drop when the degradations of test images are unknown (i.e. the case of blind super-resolution). To address these issues, we propose an online degradation estimation method for blind super-resolution. It does not rely on predefined degradations and allows the model weights to be updated according to the degradation of the test image. Specifically, it consists of two branches, the internal branch and the external branch. The internal branch could learn the specific degradation of the given test image, and the external branch could learn to super-resolve images degraded by the learned degradation. In this way, our method could customize a specific model for each test image with different degradations, and thus get more robust to various degradations. Extensive experiments on both synthetic and real-world images show that the proposed method can reconstruct more visually favorable super-resolution results and achieve state-of-the-art performance.

4. An uncertainty learning based degradation estimation method for blind super-resolution. This algorithm is mainly aimed at scenarios where the degradations of low-resolution images are unknown and the degradation types are diverse (including blur, noise, and compression). In blind super-resolution, most degradation estimation methods only predict specific degradation models or parameter values. However, due to the ill-posed nature of image degradation, accurate degradation estimation is quite challenging. To address this issue, we propose a probabilistic degradation estimator which can predict the degradation as a certain distribution rather than a single point in degradation space. Specifically, we develop a degradation regression loss with uncertainty, which could lead the estimator to shrink the possible degradation space of the test image. In this way, the degradation estimator could be more robust to estimation deviation and also synthesize more degradation-specific training samples. Thus the super-resolution model could be rapidly fine-tuned to improve reconstruction performance. Additionally, the proposed estimator is appliable for multiple degradation types, and can estimate degradation parameters of blur, noise, and compression. Extensive experiments show that the proposed degradation estimator can help the super-resolution model produce better results on both synthetic and real-world images.