毕业生知识库

    模式分类是人工智能的一个基础研究领域，是从数据中获取有效信息的一个重要手段。各种各样的分类器被相继提出，并被广泛地用于解决许多实际问题。在实际的分类问题中，存在两种常见的分类问题，一种是分类器对异常样本的鲁棒性问题，另一种是不平衡数据分类问题。因为常见的基于无界凸损失函数的分类器对异常样本的鲁棒性较差，所以异常样本可能会降低它们的分类性能。此外，由于常见分类器一般假设不同类别的样本分布近似平衡或者错分代价近似相等，所以它们在不平衡数据分类问题中的分类结果可能不满足实际要求。目前，有许多单独研究鲁棒分类问题或者单独研究不平衡数据分类问题的论文，但是同时研究这两种分类问题的论文却不多。针对上述的分类问题，本文首先提出了一组基于有界压缩变换的鲁棒损失函数用于减弱异常样本对分类器性能的影响。然后，在代价敏感学习的框架下，本文将所提出的鲁棒损失函数和非对称阶式最小二乘损失函数用于处理含有异常样本的不平衡数据分类问题。最后，在代价缺失学习的框架下，本文研究了支持向量机的互信息拒识分类准则。该拒识分类准则可以筛选出分类置信度较低的模糊样本。本文所取得的主要成果如下

 

    根据基于Correntropy的损失函数与最小二乘损失函数之间的关系，本文提出了损失函数的有界压缩变换方法。该有界压缩变换方法能够统一地将常见的无界凸损失函数，例如铰链损失函数(hinge loss function)，逻辑损失函数和指数损失函数，转化为对应的有界非凸鲁棒损失函数。与损失函数的截断方法不同，该有界压缩变换方法是对无界凸损失函数进行光滑变换，所以具有更好的优化性质。本文分别推导了基于这些鲁棒损失函数的鲁棒分类器，并采用了半二次优化方法来优化它们。半二次优化方法包含两个迭代步骤：一个是样本权重系数更新；另一个是求解凸优化问题，该凸优化问题对应一种加权分类器。通过半二次优化方法，本文将这些鲁棒分类器与已有的加权分类器建立了联系，从而能够从鲁棒损失函数的角度解释加权分类器的鲁棒特性。人造数据集和真实数据集上的实验结果表明，本文所提出的这些鲁棒分类器能够有效地降低异常样本对分类器性能的影响。

     在代价敏感学习的框架下，本文将非对称阶式最小二乘损失函数(asymmetric stagewise least square loss function)和基于有界压缩变换的鲁棒损失函数应用于含有异常样本的不平衡数据分类问题。目前，有很多单独研究鲁棒分类问题或者单独研究不平衡数据分类问题的论文，但是同时研究这两种分类问题的论文却不多。本文提出了代价敏感学习和鲁棒损失函数相结合的方法来改善分类器在含有异常样本的不平衡数据集上的分类结果。其中，代价敏感学习用于克服样本分布不平衡的问题，鲁棒损失函数用于减弱异常样本的负面影响。非对称阶式最小二乘损失函数也是一种鲁棒损失函数，该损失函数给不同类别的样本不同的损失上界和不同的margin, 从而保护正类样本。实验结果表明这种结合的方法可以改善分类器在这种数据集上的分类结果。

     在代价缺失学习的框架下，本文研究了支持向量机的互信息拒识分类准则。当遇到模糊样本时，人们可能会拒绝判断该样本的类别。将模糊样本判断为拒识类别也是一种有效地改善分类结果的方法，而被拒识的样本可以被进一步地分析或者使用其他的分类器来判断类别。互信息能够自动平衡错分样本和拒识样本的数量。在代价缺失学习的框架下，本文分析了支持向量机的互信息拒识分类准则。此外，本文还讨论了互信息拒识分类准则与代价敏感学习之间的联系，推导了等价的损失代价。该等价的损失代价可以作为代价敏感学习中损失代价的参考。本文将该支持向量机的互信息拒识分类准则应用于含有异常样本的不平衡数据分类问题。实验结果表明该互信息拒识分类准则通过拒识模糊样本，可以达到改善分类结果的目的。

    Pattern classification is a fundamental problem, which is to predict the label of an unseen sample. A variety of classifiers have been developed and used to solve a lot of practical problems. However, there are two common classification problems in practice: one is robust classification problem and the other is class imbalance problem. The common convex loss functions are not robust to outliers due to their unboundedness. Besides, common classifiers are sensitive to class imbalance since they implicitly assume that class distributions and misclassification costs are balanced. Currently, although there are many studies on outliers and class imbalance in isolation, only a few have addressed their combined influence. In this thesis, we propose several robust loss functions based on bounded squashing transform in order to reduce the detrimental influence of outliers. Then, under the framework of cost-sensitive learning, we apply the asymmetric stagewise least square loss function (\textbf{ASLS}) and the proposed robust loss functions to deal with the combined problem of outliers and class imbalance. Finally, under the framework of cost-free learning, we study the reject rule of support vector machine (\textbf{SVM}) according to mutual information (\textbf{MI}). The main contributions of this thesis are as follows

 

    According to the relationship between the Correntropy induced loss function and the least square loss function, we develop the bounded squashing transform. With this bounded squashing transform, we transform the hinge loss function, the logistic loss function and the exponential loss function into the corresponding bounded, nonconvex and robust loss functions. Different from the truncation of loss function, the bounded squashing transform keeps the smoothness of loss functions, thus the transformed loss functions have better optimization properties. We develop the robust classifiers based on the proposed robust loss functions and use the half-quadratic (\textbf{HQ}) optimization method to optimize them which has two iteration steps: one is weight updating and the other is to solve a convex optimization problem which is about a weighted classifier. Using HQ, we establish the relationship between our robust classifiers and the existing weighted classifiers. And then, we can explain the robustness of weighted classifiers from a robust loss function perspective. Experimental results on synthetic and real-world datasets confirm the effectiveness of our robust classifiers in reducing the influence of outliers.

    With cost-sensitive learning, we apply ASLS and the proposed robust loss functions to deal with the combined problem of outliers and class imbalance. Currently, although there are many works on outliers and class imbalance in isolation, only a few have studied their combined influence. In this thesis, we combine cost-sensitive learning with robust loss functions so as to improve the classification results of class imbalance problem with outliers. The cost-sensitive learning is in charge of dealing with imbalanced class distributions and the robust loss functions are used to restrain the effect of outliers. ASLS is also a robust loss function, but provides class-dependent loss bounds and class-dependent margins so as to protect the positives. We show in the experiments that this combined method can improve the classification results of such a classification problem.

    Under the framework of cost-free learning, we study the MI-based reject rule of SVM. We prefer to reject to classify a sample when we are not sure enough about its label. Abstaining classification is also a good method to improve classification results, and the rejected samples can be further analyzed or classified with another more powerful classifier. MI has a good property of automatically balancing the rejects and errors. With cost-free learning, we study the MI-based reject rule of SVM. Besides, we also establish the relationship between the MI-based reject rule and cost-sensitive learning, deriving the equivalent misclassification and reject costs. Such equivalent misclassification and reject costs can serve as the reference of cost-sensitive learning. We show the effectiveness of this MI-based reject rule of SVM in the classification problem of outliers and class imbalance.