毕业生知识库

由于应用阶段的数据分布可能与训练阶段存在差异，一个训练好的图像识别模型在实际部署时常会面临多种问题。一种典型的问题为图像质量退化导致的特征分布漂移问题。当应用阶段的图像受到质量退化影响时，其特征分布会相对清晰图像特征产生漂移，使得模型无法对退化图像进行正确的识别，产生性能下降问题。另一种典型的问题为类别增量学习场景下的开放类别问题。模型在训练过程中所能识别的类别是确定的，然而在实际应用时，新的类别会不断产生，这需要模型能够不断学习如何识别新类别。然而若是简单地对模型进行微调，则会产生灾难性遗忘问题，即模型在旧类别上的识别性能会不断下降。

针对上述问题，虽然已有相关研究，但这些工作都存在一些不足。对图像质量退化导致的特征分布漂移问题，现有的研究工作大多集中于对退化图像进行质量增强再进行识别。然而，质量增强的目的为提升视觉质量而非识别性能，因此其对模型的性能提升有限，并且无法处理更复杂的复合退化情形。对类别增量学习场景下的开放类别问题，现有的工作在学习新类别时过分关注了如何更好地区分旧类别与新类别，而忽视了如何为未来类别的学习预留能力。因此，本文将分别对图像质量退化导致的特征分布漂移和类别增量学习场景下的开放类别问题，通过对图像特征进行增强，对模型特征空间进行微调或是提高其适应将来类别分布的能力，提升图像识别模型的性能。本文的主要内容和贡献点如下：

(1) 针对图像质量退化导致的特征分布漂移问题，本文设计了一种图像特征矫正模块（Robust Feature Rectification, ROFER）。该模块可对图像受到的退化类型及其强度进行分类和回归，然后通过一个全卷积网络将图像的特征根据预测的退化类型矫正至清晰特征。该模块可同时处理多种退化，包括常见的模糊、噪声、对比度影响，并且能够轻松地插入到预训练的图像识别模型中，通过对模型的特征空间进行微调，提升其在退化图像上的识别性能。同时，通过引入束搜索策略以寻找退化的复合顺序，该模块可通过迭代的方式处理多种退化复合的情况。实验结果表明，ROFER能有效提升模型在受单一退化或复合退化影响的图像上的识别性能。

(2) 针对类别增量学习场景下的开放类别问题，本文首先分析了灾难性遗忘问题产生的原因之一为已见类别的过度坍缩，即模型会倾向于将一已见类别与其相邻区域映射至特征空间的同一位置。在该分析以及理论推导的基础之上，本文设计了一种特征增强方法（Preventing Over-Collapse, POC）。该方法通过学习一组仿射变换，使得该组仿射变换在作用到已见类别上时，会产生位于其相邻区域内的样本。在获得已见类别相邻区域内的样本后，模型的训练损失中将增加一项将已见类别与其相邻区域进行分类的损失，通过该方式，防止模型将已见类别与相邻区域映射至特征空间的同一位置，为未来类别的学习保留能力。同时，为了保证模型在已见类别上的泛化性，本文引入了一项对比损失，使得已见类别样本的特征与其相邻区域内样本的特征之间相互接近，进一步提升POC的有效性。实验结果表明，POC能和现有类别增量学习方法相结合，有效提升其性能。

 Since the data distribution in the application stage may be different from that in the training stage, a trained image recognition model will face various problems when it is actually deployed. A typical problem is the feature distribution drift caused by image degradation. When the image in the application stage is affected by degradations, its feature distribution will drift relative to the clear image features, making the model unable to correctly recognize the degraded image, resulting in performance degradation. Another typical problem is the open class problem in the class-incremental learning scenario. The classes that the model can recognize during the training process are determined. However, in actual applications, new classes will continue to be generated, which requires the model to be able to continuously learn how to recognize new categories. However, if simply fine-tune the model, the model will suffer from the catastrophic forgetting problem, where the model's recognition performance on old categories will continue to decline.

Although there have been relevant studies on the above two problems, they have some shortcomings. Regarding the feature distribution drift problem caused by image degradation, most of the existing work focuses on improving the quality of degraded images for recognition. However, the purpose of quality enhancement is to improve visual quality rather than recognition performance, so its ability is limited and cannot handle more complex composite degradation situations. Regarding the open class problem in the class-incremental learning scenario, existing work focuses too much on how to better distinguish old classes from new classes when learning new tasks, but ignores how to reserve capabilities for future class learning. Therefore, this thesis will respectively address the feature distribution drift caused by image quality degradation and the open class problem in the class-incremental learning scenario. By enhancing the image features, the model's feature space is fine-tuned or its ability to adapt to future class distribution is improved. The main contents and contributions of this article are as follows:

(1) To address the problem of feature distribution drift caused by image degradation, this thesis designs an image feature rectification module (ROFER), which can classify and regress the degradation type and intensity of the image, and then use a fully convolutional network to rectify the image features according to the predicted degradation type to clear features. This module can handle multiple degradations simultaneously, including blur, noise, and low contrast, and can be easily plugged into pre-trained image recognition models to improve their recognition performance on degraded images. At the same time, by introducing a beam search strategy to find degradation composition orders, this module can handle composite degradations in an iterative manner. Our experiments show that ROFER can effectively improve the recognition performance on images affected by single or composite degradations. 

(2) For open class problem in class-incremental learning scenarios, this thesis first analyzes that one of the causes of the catastrophic forgetting problem is over-collapse of seen classes, that is, the model tends to map a seen class and its adjacent regions to the same position in the feature space. Based on this analysis and theoretical derivation, this thesis designs a feature enhancement method (POC), which learns a set of affine transformations so that when they are applied to the learned categories, they will produce samples located within its adjacent regions. After obtaining samples in the adjacent regions of the seen classes, a loss for classifying the seen class and the adjacent regions will be added to the classification loss. In this way, the model is prevented from mapping the seen classes and the adjacent regions to the same position in feature space, improving its the ability for future class learning. POC can be combined with any class-incremental learning methods to improve its performance. At the same time, in order to ensure the generalization of the model on the seen classes, we introduce a deterministic contrastive loss to make the features close to each other, further improving the effectiveness of the method. The designed experiments show that POC effectively improves the performance of class-incremental learning methods.